tsdr.git
3 years agoAdd INFO.yaml for tsdr 72/91272/1 master
Anil Belur [Sun, 12 Jul 2020 02:55:48 +0000 (12:55 +1000)]
Add INFO.yaml for tsdr

Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I9ea69b7f8c8529357f29b32185adb0a876775eb0

5 years agoAdd kafka producer 16/75816/16
Matcauf [Thu, 6 Sep 2018 14:48:33 +0000 (10:48 -0400)]
Add kafka producer

Add kafka producer that is using json by default.

Change-Id: I4f53d7fc82d32ca33293181fdff6c20315929b09
Signed-off-by: Matcauf <matthieu.cauffiez@bell.ca>
5 years agoMove persistence-hsql BP xml to OSGI-INF 32/80032/1
Tom Pantelis [Wed, 30 Jan 2019 16:26:14 +0000 (11:26 -0500)]
Move persistence-hsql BP xml to OSGI-INF

Change-Id: I05fc75c03210f72984c9057e092f8c70ea3cff98
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoBump versions 95/79995/2
Tom Pantelis [Tue, 29 Jan 2019 16:38:52 +0000 (11:38 -0500)]
Bump versions

odlparent 4.0.9,  mdsal 3.0.6

Change-Id: Ia44a1b0d711a45c2393aa3d60a1874e65b7dd93d
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Yang notification collector UTs and refactoring"
Scott Melton [Wed, 7 Nov 2018 20:50:04 +0000 (20:50 +0000)]
Merge "Yang notification collector UTs and refactoring"

5 years agoYang notification collector UTs and refactoring 73/77473/6
Tom Pantelis [Sun, 4 Nov 2018 03:18:45 +0000 (23:18 -0400)]
Yang notification collector UTs and refactoring

JIRA: TSDR-90
Change-Id: Ia9855bebb179c28e304b0472561c9e0a5976bedf
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Implement netflow IPFIX protocol"
Scott Melton [Tue, 6 Nov 2018 06:11:48 +0000 (06:11 +0000)]
Merge "Implement netflow IPFIX protocol"

5 years agoAdd YangCollector 55/76655/9
Matcauf [Thu, 4 Oct 2018 16:07:19 +0000 (18:07 +0200)]
Add YangCollector

Add a new feature named
odl-tsdr-yang-notification-collector
this feature will collect and send yang notifation
to a back end

JIRA: TSDR-90
Change-Id: I580a9f49fcbc775e4a640e0d8abed1675d49c6b7
Signed-off-by: Matcauf <mcauffiez@inocybe.com>
5 years agoImplement netflow IPFIX protocol 59/77159/13
Tom Pantelis [Sun, 21 Oct 2018 01:17:02 +0000 (21:17 -0400)]
Implement netflow IPFIX protocol

As per https://tools.ietf.org/search/rfc7011. Successor to and based
on netflow v9.

Change-Id: I0f694c31c5aa15210722ec2b3bdd27ad6c5f6eed
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoNeon MRI version bumps 72/77472/4
Tom Pantelis [Sat, 3 Nov 2018 15:23:33 +0000 (11:23 -0400)]
Neon MRI version bumps

Change-Id: I5189f2e61f19362306f53980c38eea9e9e88f2c2
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Use BatchingLogCollector in TSDRNetflowCollectorImpl"
Scott Melton [Tue, 16 Oct 2018 03:27:34 +0000 (03:27 +0000)]
Merge "Use BatchingLogCollector in TSDRNetflowCollectorImpl"

5 years agoMerge changes Ia1286f0b,I04b3f64f
Tom Pantelis [Mon, 15 Oct 2018 19:25:42 +0000 (19:25 +0000)]
Merge changes Ia1286f0b,I04b3f64f

* changes:
  Cache netflow records for missing template
  Use netflow packet time as log record timestamp

5 years agoUse BatchingLogCollector in TSDRNetflowCollectorImpl 46/76946/2
Tom Pantelis [Fri, 12 Oct 2018 15:40:27 +0000 (11:40 -0400)]
Use BatchingLogCollector in TSDRNetflowCollectorImpl

Change-Id: Ieb0b2d5b1313f14cdb83dbcff3f1edb4c13ca819
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Netflow template key should include the exporter's IP"
Scott Melton [Thu, 11 Oct 2018 19:00:20 +0000 (19:00 +0000)]
Merge "Netflow template key should include the exporter's IP"

5 years agoCache netflow records for missing template 09/76809/3
Tom Pantelis [Tue, 9 Oct 2018 20:33:41 +0000 (16:33 -0400)]
Cache netflow records for missing template

https://www.ietf.org/rfc/rfc3954.txt states that a collector
should store data records if the template record(s) haven't been
received yet when the data records are received and decode them after
the template record(s) are received. The exporter is supposed to
send templates before any data records that reference them however
the UDP protocol doesn't guarantee packets will arrive at the
destination in the same order that the sender put them on the wire.

Added a MissingTemplateCache to cache NetflowV9PacketParser instances
per template key when a missing template is encountered. On subsequent
packets, the cache is checked for prior missing templates and, if now
present, the parsers are invoked to continue parsing. If a template isn't
received within a minute, the parsers are evicted from the cache and
remaining records are dropped.

Change-Id: Ia1286f0bb8cd00baf84527d53855035f08a935e5
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Add netflow Options templates and data records support"
Scott Melton [Wed, 10 Oct 2018 20:59:13 +0000 (20:59 +0000)]
Merge "Add netflow Options templates and data records support"

5 years agoUse netflow packet time as log record timestamp 97/76697/2
Tom Pantelis [Fri, 5 Oct 2018 13:01:22 +0000 (09:01 -0400)]
Use netflow packet time as log record timestamp

Instead of including it in the record attrs. I don't think
reporting the collector's timestamp as well would be useful
to end users as the collector is essentially a pass-thru.

Change-Id: I04b3f64f85ea18266de48d424bda9226fd010ac4
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoNetflow template key should include the exporter's IP 45/76645/2
Tom Pantelis [Thu, 4 Oct 2018 14:08:11 +0000 (10:08 -0400)]
Netflow template key should include the exporter's IP

According to https://www.ietf.org/rfc/rfc3954.txt,
"Template IDs are unique per Exporter and per Observation Domain."
and "the Collector SHOULD maintain the following for all
the current Template Records and Options Template Records: Exporter,
Observation Domain, Template ID, Template Definition, Last Received.".

Change-Id: I077f5e2b85b48c7feb8272989d79677e5fb06ed2
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoAdd netflow Options templates and data records support 20/76620/4
Tom Pantelis [Wed, 3 Oct 2018 20:22:49 +0000 (16:22 -0400)]
Add netflow Options templates and data records support

As per https://www.ietf.org/rfc/rfc3954.txt. Also refactored the
template classes for code reuse between data flowset and options
templates.

The v5 and v9 classes were split into separate packages for clarity.

Change-Id: I4abc94affe26544b0a1d9f7ed95de77143f341a7
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Add netflow v9 template cache"
Scott Melton [Wed, 3 Oct 2018 22:10:48 +0000 (22:10 +0000)]
Merge "Add netflow v9 template cache"

5 years agoMerge "Refactor NetflowPacketParser"
Scott Melton [Wed, 3 Oct 2018 22:10:12 +0000 (22:10 +0000)]
Merge "Refactor NetflowPacketParser"

5 years agoMerge "Improve NetflowPacketParser"
Tom Pantelis [Wed, 3 Oct 2018 02:17:41 +0000 (02:17 +0000)]
Merge "Improve NetflowPacketParser"

5 years agoAdd netflow v9 template cache 87/76587/1
Tom Pantelis [Tue, 2 Oct 2018 22:28:53 +0000 (18:28 -0400)]
Add netflow v9 template cache

Template records aren't sent in every export packet so
they must be cached in order to process subsequent
dataflow sets.

Change-Id: I5148bef61cd07151e14f8eff71baa31630c5a700
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRefactor NetflowPacketParser 66/76566/1
Tom Pantelis [Tue, 2 Oct 2018 15:50:49 +0000 (11:50 -0400)]
Refactor NetflowPacketParser

Convert NetflowPacketParser to an interface and separate
the implementations into their own class files. Also added
a NetflowPacketParserFactory that will be used to hold
data across NetflowPacketParser instances.

Change-Id: Ia2d84bcd0e65d8b68fe0d38b9e568f47c47a9056
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Use QueuedNotificationManager in SyslogDatastoreManager"
Tom Pantelis [Mon, 1 Oct 2018 13:57:35 +0000 (13:57 +0000)]
Merge "Use QueuedNotificationManager in SyslogDatastoreManager"

5 years agoImprove NetflowPacketParser 69/76469/6
Tom Pantelis [Thu, 27 Sep 2018 03:02:30 +0000 (23:02 -0400)]
Improve NetflowPacketParser

Separated out v5 and v9 NetflowPacketParser classes
for clarity with NetflowPacketParser as the abstract base.

Instead of using hard-coded offsets into the byte stream
which is error prone, the current position is tracked and
updated automatically.

The v5 code was mostly correct except the sampling_interval field
is 14 bits and not 6.

The v9 code was incorrect for several reasons wrt
https://www.ietf.org/rfc/rfc3954.txt, in particular:

 - It assumed there is only one template when parsing a data flow set.

 - If there was no template, it assumed a particular template format.
   A data flow set cannot be parsed without a corresponding template.

 - When parsing a template, the template ID and field count were not
   parsed and thus were interpreted as a field record.

The code also assumes a template flow set is always in the same packet
as the data flow sets but templates are mostly sent once and are supposed
to be cached in the collector to be used to decode subsequent data flow
set packets. This will be addressed in another patch.

Change-Id: I74c0f3bdae266418b34195e25aa35167fb43a5df
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoImprove TSDRNetflowCollectorImplTest 48/76448/1
Tom Pantelis [Tue, 25 Sep 2018 19:05:01 +0000 (15:05 -0400)]
Improve TSDRNetflowCollectorImplTest

Verify log records were stored.

Change-Id: Ia17e4cef33982349a65985beded3086cd028f1f3
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoUse QueuedNotificationManager in SyslogDatastoreManager 30/76430/4
Tom Pantelis [Mon, 24 Sep 2018 22:27:12 +0000 (18:27 -0400)]
Use QueuedNotificationManager in SyslogDatastoreManager

This provides a queue per client callback backed by a shared
thread pool that preserves ordering of incoming notifications
and allows notifications to be batched per client.

Change-Id: I7ea22abda38e570a60c356b94caa1dcdd158aa62
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoSimplify SyslogFilter 65/76365/1
Tom Pantelis [Sat, 22 Sep 2018 14:34:33 +0000 (10:34 -0400)]
Simplify SyslogFilter

The interface was simplified to just the matches(...) API. The
getPersistenceDestination API was not used and not clearly defined,
specifically as to what the return int value signifies. For the
filterAndParseSyslog API, it seems the "filter" aspect should already
be handled by matches - it also seems that building the TSDRLogRecord
shouldn't be related to the filtering and should be consistent. There is
only one implementation of SyslogFilter that basically is a no-op so it's
hard to know what the original intention was.

Also, the map in the SyslogFilterManager was changed from SyslogFilter[] to
SyslogFilter. It's unclear why this was an array - the code only expected one
element and, even if it allowed multiple, the applyFilters method only returns
a single TSDRLogRecord. So again, unclear as to what the original intention was.

Change-Id: I8264056f0bb48d3edbbf6b918b0dcfbccffd6107
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoCleanup MessageFilter 14/76314/1
Tom Pantelis [Thu, 20 Sep 2018 17:20:21 +0000 (13:20 -0400)]
Cleanup MessageFilter

- Precompile Patterns for efficiency

- Rename equals(Message) to matshes(Message) to avoid confusion
  with equals(Object)

- Remove the FilterBuilder inner class - it does not really follow
  the builder pattern and isn't needed - replace with static
  MessageFilter.from(Filter).

Change-Id: I64b014b44ba5e4a55de066fa892dfa5184973891
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMove Facility and Severity enums to yang 13/76313/1
Tom Pantelis [Thu, 20 Sep 2018 16:34:46 +0000 (12:34 -0400)]
Move Facility and Severity enums to yang

Instead of defining them in the Message class. This enables the
enums to be used in the meta-filter grouping instead of using the
internal enum ordinals.

Change-Id: I6ac4ad34eb6fa113472ac22261f28b1ce7373fdf
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRefactor/Simplify SyslogDatastoreManager 83/76283/3
Tom Pantelis [Wed, 19 Sep 2018 17:46:05 +0000 (13:46 -0400)]
Refactor/Simplify SyslogDatastoreManager

as per TSDR-88.

Change-Id: Ic32ef265bc3246c3416936fd737f3871f564d51c
JIRA: TSDR-88
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoSimplify TSDRSyslogCollectorImpl 49/75849/3
Tom Pantelis [Fri, 7 Sep 2018 20:13:20 +0000 (16:13 -0400)]
Simplify TSDRSyslogCollectorImpl

Use the AbstractBatchingLogCollector for queueing/logging syslog
messages instead of a dedicated thread for simplification. Also hide
the actual message queue behind a MessageQueue interface for
enqueuing.

Change-Id: I5ef5aa5b95b087df971fa77850eca2f8aa5ef816
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Add AbstractBatchingLogCollector"
Tom Pantelis [Tue, 18 Sep 2018 12:51:59 +0000 (12:51 +0000)]
Merge "Add AbstractBatchingLogCollector"

5 years agoMerge "Precompile Patterns for the restconf filter checking"
Scott Melton [Tue, 18 Sep 2018 03:11:17 +0000 (03:11 +0000)]
Merge "Precompile Patterns for the restconf filter checking"

5 years agoAdd AbstractBatchingLogCollector 91/76191/1
Tom Pantelis [Mon, 17 Sep 2018 16:57:53 +0000 (12:57 -0400)]
Add AbstractBatchingLogCollector

Extracted from BatchingLogCollector - genericizes the type of
the incoming objects that are queued and transforms them to
TSDRLogRecords via an abstract method. BatchingLogCollector
specifies TSDRLogRecord as the incoming type.

Change-Id: Ib255c0c9cf7c800c396dd470844c37ac9e7fadd6
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoPrecompile Patterns for the restconf filter checking 93/76093/2
Tom Pantelis [Fri, 14 Sep 2018 15:00:01 +0000 (11:00 -0400)]
Precompile Patterns for the restconf filter checking

Compiling regex Patterns on the fly per check is expensive
so precompile them in TSDRRestconfCollectorConfig.

Also sreamlined and simplified the parsing code in
TSDRRestconfCollectorConfig.

Change-Id: I2bbbdebf60538684238f9d68de62e949044fa782
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Advertise the TSDRRestconfCollectorFilter instance as OSGi service"
Tom Pantelis [Fri, 14 Sep 2018 14:59:19 +0000 (14:59 +0000)]
Merge "Advertise the TSDRRestconfCollectorFilter instance as OSGi service"

5 years agoMerge "migrate to mdsal.binding.api.DataBroker"
Tom Pantelis [Thu, 13 Sep 2018 23:03:19 +0000 (23:03 +0000)]
Merge "migrate to mdsal.binding.api.DataBroker"

5 years agomigrate to mdsal.binding.api.DataBroker 64/76064/2
Matcauf [Thu, 13 Sep 2018 14:47:53 +0000 (10:47 -0400)]
migrate to mdsal.binding.api.DataBroker

Migrate Syslog collector to mdsal.binding.api.DataBroker

Change-Id: Icfaca3045adaae4b71d8658d746a464b14ab0e8c
Signed-off-by: Matcauf <mcauffiez@inocybe.com>
5 years agomigrate to mdsal.binding.api.DataBroker 46/76046/7
Matcauf [Wed, 12 Sep 2018 21:00:08 +0000 (17:00 -0400)]
migrate to mdsal.binding.api.DataBroker

Migrate Openflow collector to
 mdsal.binding.api.DataBroker

Change-Id: I29bf2b69f2bed92323e86185dedb95bf73c1d249
Signed-off-by: Matcauf <mcauffiez@inocybe.com>
5 years agoAdvertise the TSDRRestconfCollectorFilter instance as OSGi service 61/76061/2
Tom Pantelis [Wed, 12 Sep 2018 11:55:38 +0000 (07:55 -0400)]
Advertise the TSDRRestconfCollectorFilter instance as OSGi service

This allows the dependencies to be properly injected instead of
set and stored statically.

Change-Id: Iaa1f6693bb7289ed966d5f3fa96ca8760e5e1c46
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge " Add log collector"
Tom Pantelis [Wed, 12 Sep 2018 00:31:02 +0000 (00:31 +0000)]
Merge " Add log collector"

5 years ago Add log collector 94/75594/19
Matcauf [Thu, 30 Aug 2018 20:59:50 +0000 (16:59 -0400)]
 Add log collector

Add a odl log collector, this feature will collect
odl logs and send them to the backend, it is based on
tsdr restconf collector

Change-Id: If37661e488e3b6ff72547d92d50fba00cff91201
Signed-off-by: Matcauf <mcauffiez@inocybe.com>
5 years agoMerge "Migrate TSDR docs to the TSDR project"
Scott Melton [Sun, 9 Sep 2018 19:29:08 +0000 (19:29 +0000)]
Merge "Migrate TSDR docs to the TSDR project"

5 years agoMerge "Bootstrap TSDR documentation"
Scott Melton [Sun, 9 Sep 2018 18:55:29 +0000 (18:55 +0000)]
Merge "Bootstrap TSDR documentation"

5 years agoMerge "Simplify ControllerMetricCollector"
Scott Melton [Sun, 9 Sep 2018 06:47:35 +0000 (06:47 +0000)]
Merge "Simplify ControllerMetricCollector"

5 years agoMerge "Improve ControllerMetricCollector UTs"
Scott Melton [Sun, 9 Sep 2018 05:35:55 +0000 (05:35 +0000)]
Merge "Improve ControllerMetricCollector UTs"

5 years agoMerge "Use SchedulerService in ControllerMetricCollector"
Scott Melton [Sun, 9 Sep 2018 04:37:23 +0000 (04:37 +0000)]
Merge "Use SchedulerService in ControllerMetricCollector"

5 years agoMigrate TSDR docs to the TSDR project 59/75859/2
Thanh Ha [Sun, 9 Sep 2018 00:00:29 +0000 (20:00 -0400)]
Migrate TSDR docs to the TSDR project

Move the TSDR docs into the TSDR project. Once available we
can activate TSDR project specific documentation builds and
link from ODL main docs to the TSDR project.

Issue: DOCS-69
Change-Id: Id44e0fce49ed72d8034b65caa222285044fc7368
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoBootstrap TSDR documentation 58/75858/2
Thanh Ha [Sat, 8 Sep 2018 23:51:58 +0000 (19:51 -0400)]
Bootstrap TSDR documentation

Issue: DOCS-69
Change-Id: Ib87242fd465aacb9c72d433fc61615ad1da710ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSimplify ControllerMetricCollector 57/75857/2
Tom Pantelis [Sat, 8 Sep 2018 17:22:59 +0000 (13:22 -0400)]
Simplify ControllerMetricCollector

The CpuDataCollector interface returns Optional data so we really
don't need the CpuDataCollector itself to be optional in the
ControllerMetricCollector. If there's no real implementation of
CpuDataCollector available then use a null instance that returns
empty Optionals.

Also use OptionalDouble instead of Optional<Double> for efficiency.

Also streamlined the code in ControllerMetricCollector.

Change-Id: I7284cc1959e73781e9971dcc0dcc228d043b28ac
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoImprove ControllerMetricCollector UTs 56/75856/1
Tom Pantelis [Sat, 8 Sep 2018 16:28:16 +0000 (12:28 -0400)]
Improve ControllerMetricCollector UTs

Added end-to-end tests and verification.

Change-Id: Ied5e850c0ac641a2da5ae3f32a47c72cfb555dc3
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoUse SchedulerService in ControllerMetricCollector 55/75855/1
Tom Pantelis [Sat, 8 Sep 2018 15:19:48 +0000 (11:19 -0400)]
Use SchedulerService in ControllerMetricCollector

instead of a dedicated thread.

Change-Id: I5d5b98516ca294cf2e3cdd7d95250196570573f4
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoConsolidate TSDRSyslogCollectorImpl unit tests 46/75846/2
Tom Pantelis [Fri, 7 Sep 2018 16:55:25 +0000 (12:55 -0400)]
Consolidate TSDRSyslogCollectorImpl unit tests

The UTs were spread out over several test classes - they're
now consolidated into a TSDRSyslogCollectorImplTest class
which eliminates the duplicate setup and makes it easier
to run all TSDRSyslogCollectorImpl UTs.

Also sped up the tests by eliminating arbitrary sleeps and
improving startup and shutdown sequencing.

Change-Id: Ia41b156320e82702ac7e3912c355f5a9da2b2700
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRemove syslog config-threadpool RPC 40/75840/1
Tom Pantelis [Fri, 7 Sep 2018 11:57:15 +0000 (07:57 -0400)]
Remove syslog config-threadpool RPC

This RPC isn't really useful b/c it modifies config params
directly in the running ThreadPoolExecutor but does not persist
the new settings to config DS. These config params should be set
via updating the syslog-collector-config yang model.

Change-Id: I957d4b004ef0384aef94b971014e5e0311432407
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoMerge "Improve UDPServerTest"
Scott Melton [Fri, 7 Sep 2018 04:48:00 +0000 (04:48 +0000)]
Merge "Improve UDPServerTest"

5 years agoMerge "Consolidate SyslogDatastoreManager unit tests"
Scott Melton [Fri, 7 Sep 2018 03:21:56 +0000 (03:21 +0000)]
Merge "Consolidate SyslogDatastoreManager unit tests"

5 years agoMerge "Bump versions by x.(y+1).z for the next dev cycle"
Scott Melton [Fri, 7 Sep 2018 01:42:49 +0000 (01:42 +0000)]
Merge "Bump versions by x.(y+1).z for the next dev cycle"

5 years agoImprove UDPServerTest 23/75823/2
Tom Pantelis [Thu, 6 Sep 2018 23:54:43 +0000 (19:54 -0400)]
Improve UDPServerTest

Removed the arbitrary 10s sleep in lieu of waiting for
messages. Also use a mock SyslogDatastoreManager to avoid
exceptions due to a mock DataBroker - in this case it's enough
to just verify the execute method is called.

Also renamed to SyslogUDPServer to mathc the class name.

Change-Id: Icb02daf0d77d65db33dc0d61f6fb5765f1c1846c
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoConsolidate SyslogDatastoreManager unit tests 21/75821/2
Tom Pantelis [Thu, 6 Sep 2018 23:07:42 +0000 (19:07 -0400)]
Consolidate SyslogDatastoreManager unit tests

The UTs were spread out over several test classes - they're
now consolidated into a SyslogDatastoreManagerTest class
which eliminates the duplicate setup and makes it easier
to run all SyslogDatastoreManager UTs.

Change-Id: If1e138e8db64761765a58b9d7644f0765a5d74ea
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRefactor HBaseDataStoreContext 75/75775/2
Tom Pantelis [Wed, 5 Sep 2018 19:50:40 +0000 (15:50 -0400)]
Refactor HBaseDataStoreContext

- Removed setters and initialized fields in constructor from a Properties
  instance. This allows fields to be final.

- Removed the static commonHbasePropertiesMap. The only property stored
  was createTableRetryInterval - make it a field instead.

- Added an accessor in HBaseDataStoreFactory to obtain the
  HBaseDataStoreContext instance.

Change-Id: I5049d6f9313c6e0d9907dcf0491499cc20cae177
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRemove static HBaseDataStore instance 74/75774/2
Tom Pantelis [Wed, 5 Sep 2018 18:53:52 +0000 (14:53 -0400)]
Remove static HBaseDataStore instance

Make HBaseDataStoreFactory a proper singleton and inject
instead of obtaining the HBaseDataStore instance statically.

Change-Id: I85afe35f701d42efae4ec7dc5525c1d68d82bc32
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRemove Task class 69/75769/1
Tom Pantelis [Wed, 5 Sep 2018 16:40:32 +0000 (12:40 -0400)]
Remove Task class

The Task basically wraps Runnable and adds a setScheduledFuture method.
setScheduledFuture is called by the SchedulerService after the task is
scheduled however this is problematic as it's possible for the scheduled
task to run before setScheduledFuture is called (especially if initial delay
is 0), which could lead to unexpected behavior. There are no longer any
users of the ScheduledFuture so we can eliminate it from the API which then
renders the Task class unnecessary so remove it altogther.

Change-Id: I879c6430a1563886c8487db64d4b4794c8268afe
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoBump versions by x.(y+1).z for the next dev cycle 15/75715/1
Anil Belur [Tue, 4 Sep 2018 11:57:11 +0000 (17:27 +0530)]
Bump versions by x.(y+1).z for the next dev cycle

Change-Id: I9422715861da6e3bdab0dbb8f527fb2f4687eb73
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRefactor TsdrRHBasePersistenceServiceImpl and CreateTableTask 51/75651/3
Tom Pantelis [Fri, 31 Aug 2018 22:39:25 +0000 (18:39 -0400)]
Refactor TsdrRHBasePersistenceServiceImpl and CreateTableTask

Refactored the relationship between these 2 classes and added
retries for data store operations on occurrence of a
TableNotFoundException. Previously if a TableNotFoundException
occured, a new CreateTableTask (if one was not already running)
was created but the current operation just silently exited
without retrying the operation when the CreateTableTask completed.

Also data store operations now wait for the current CreateTableTask
to complete.

The CreateTableTask now reschedules itself each time if necessary
after execution instead of scheduling at a fixed rate and cancelling
itself. This avoids needing the ScheduledFuture injected which is
problematic.

Beefed up UTs...

Change-Id: I881e849ec578843c2bb6d6e5ab0ab464653dc0bb
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoRemove static SchedulerService instance 02/75602/4
Tom Pantelis [Fri, 31 Aug 2018 02:50:23 +0000 (22:50 -0400)]
Remove static SchedulerService instance

Make it a proper singleton with an interface and implementation
advertised as an OSGi service. SchedulerService was changed to
an interface. The SchedulerServiceImpl was put in a new bundle,
tsdr-persistence-impl, to separate the impl from the interface
and to avoid the BP xml from being scanned by other bundles that
use the maven-blueprint-bundle.

The SchedulerService methods were also changed to return
ListenableScheduledFuture so users can wait async for completion.

Change-Id: I7dea6516e117ae3e64ec240c5c8474c702775530
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoImprove CollectorSpiImplTest 95/75595/1
Tom Pantelis [Thu, 30 Aug 2018 21:18:25 +0000 (17:18 -0400)]
Improve CollectorSpiImplTest

Add verification and failure test cases.

Also modified CollectorSPIImpl to use stream transformation.

Change-Id: I7b84e1eeea7d689ab46bf0fa41dcae0a2447eabc
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoBUG feature odl-controller-metrics-collector 72/75472/5
Matcauf [Fri, 24 Aug 2018 12:36:06 +0000 (14:36 +0200)]
BUG feature odl-controller-metrics-collector

the feature odl-tsdr-controller-metrics-collector was
not possible to start, this patch solves the issue

Change-Id: Iec43bcf67bcc9798d6f21d5733a563acec39877f
Signed-off-by: Matcauf <mcauffiez@inocybe.com>
5 years agoRemove the loading of odl-mdsal-models 87/75487/3
scottmelton [Sun, 26 Aug 2018 23:17:11 +0000 (16:17 -0700)]
Remove the loading of odl-mdsal-models

JIRA:TSDR-77
JIRA:MDSAL-363
JIRA:TSC:128

Change-Id: I9a0706f4f8629c13a23d8b549a451f034afbad5b
Signed-off-by: scottmelton <smelton2@uccs.edu>
5 years agoMerge "Bump to odlparent 3.1.3"
Tom Pantelis [Sat, 4 Aug 2018 02:13:55 +0000 (02:13 +0000)]
Merge "Bump to odlparent 3.1.3"

5 years agoBump to odlparent 3.1.3 77/74777/1
scottmelton [Thu, 2 Aug 2018 20:55:06 +0000 (13:55 -0700)]
Bump to odlparent 3.1.3

JIRA:TSDR-76
JIRA:TSDR-75

Change-Id: I1019251d521f00c6d66a25b937e2aa8704aec95b
Signed-off-by: scottmelton <smelton2@uccs.edu>
5 years agoHandle invalid request formats 56/74556/7
scottmelton [Fri, 27 Jul 2018 02:21:20 +0000 (19:21 -0700)]
Handle invalid request formats

JIRA: TSDR-78
Add code to provide error handling and messaging on REST queries.

Change-Id: I180ca82bf9a88ab356ef8ef464ec95cf10de521f
Signed-off-by: scottmelton <smelton2@uccs.edu>
5 years agoBump netty to 3.10.6 90/71290/6
Robert Varga [Wed, 25 Apr 2018 08:28:41 +0000 (10:28 +0200)]
Bump netty to 3.10.6

3.10.6 is the final release of netty-3, upgrade to it -- it works
for akka, it should work for cassandra, too.

Change-Id: I6a8f09c820f3292b66f7a89eea2b4f381fc90c76
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
5 years agoConvert to jersey 2 40/72740/2
Tom Pantelis [Wed, 6 Jun 2018 17:47:23 +0000 (13:47 -0400)]
Convert to jersey 2

Change-Id: I434c45810e8c035ed8f93a7b85edaf9cfc6be357
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
5 years agoAdjust to getKey/getAugmentation signature update 29/72729/2
Tom Pantelis [Wed, 6 Jun 2018 14:33:18 +0000 (10:33 -0400)]
Adjust to getKey/getAugmentation signature update

- getKey() -> key()
- setKey() -> withKey()
- getAugmentation() -> augmentation()

Change-Id: Ied43ebfb507534102ab8620b386f8e3a390a2d3d
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoAdjust to RPC method signature update 00/72100/5
Tom Pantelis [Sat, 19 May 2018 03:51:38 +0000 (23:51 -0400)]
Adjust to RPC method signature update

Change-Id: I85a95a286b70a07d73ec7cd7b423204d5861b14a
JIRA: TSC-99
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoAllow records in TSDR to have brackets inside their keys 61/71661/4
AbdulRahman AlHamali [Wed, 2 May 2018 10:42:56 +0000 (13:42 +0300)]
Allow records in TSDR to have brackets inside their keys

This was done by changing the way we read data from the database. The
storing did not have trouble, but reading looked for ']' to determine
the end of the key. So, I made sure that I got the last ']' instead
of one in the middle

Change-Id: I1d5697d12818b4c9e5cd53e69748ec51b001b785
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoFix odl-tsdr-restconf-collector SFT failure 71/71671/3
Tom Pantelis [Wed, 2 May 2018 20:49:12 +0000 (16:49 -0400)]
Fix odl-tsdr-restconf-collector SFT failure

 org.osgi.service.blueprint.container.ComponentDefinitionException: Error when instantiating bean .component-1 of class org.opendaylight.restconf.nb.bierman02.web.noauth.WebInitializer
02:49:35  at org.apache.aries.blueprint.container.BeanRecipe.wrapAsCompDefEx(BeanRecipe.java:361)
...
Caused by: java.lang.IllegalStateException: Http context already used. Context params can be set/changed only before first usage
02:49:35  at org.ops4j.pax.web.service.internal.HttpServiceStarted.setContextParam(HttpServiceStarted.java:707)

Removed the features-restconf and features-aaa dependencies from the pom.
These are feature repos and not intended to be installed as features.

Change-Id: Ia1a8a5066d6e301adbc8812d8d91d0e03ea38716
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoMerge "Fixed local karaf distribution"
Tom Pantelis [Wed, 25 Apr 2018 22:23:41 +0000 (22:23 +0000)]
Merge "Fixed local karaf distribution"

6 years agoFixed local karaf distribution 94/71294/1
AbdulRahman AlHamali [Wed, 25 Apr 2018 09:01:20 +0000 (12:01 +0300)]
Fixed local karaf distribution

Change-Id: I5e453ed5def330fdee0d8979e6d88c701fbd36a1
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoRemove unused version properties 89/71289/1
Robert Varga [Wed, 25 Apr 2018 08:27:39 +0000 (10:27 +0200)]
Remove unused version properties

There is a bunch of unused version declarations in parent,
remove them.

Change-Id: I62c12e05ac0efa83631a59315518cd6e64738e8f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
6 years agoBump to odlparent 3.1.0 and yangtools 2.0.3 93/69893/4
Stephen Kitt [Thu, 22 Mar 2018 17:24:31 +0000 (18:24 +0100)]
Bump to odlparent 3.1.0 and yangtools 2.0.3

Change-Id: I6405e134e85a53a6fbd929c3a62905979e2ab6eb
Signed-off-by: Stephen Kitt <skitt@redhat.com>
6 years agoConvert NBI to use new web registration API 63/70063/1
Tom Pantelis [Sun, 25 Mar 2018 20:27:29 +0000 (16:27 -0400)]
Convert NBI to use new web registration API

Change-Id: I644dac646f09a261ddd285aca744ca4318b9dbb9
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoUse AAAShiroFilter instead of AAAFilter 54/70054/1
Tom Pantelis [Sat, 24 Mar 2018 20:08:38 +0000 (16:08 -0400)]
Use AAAShiroFilter instead of AAAFilter

AAAFilter is really only for restconf and needs to be activated
by insalling the aaa-shiro-act bundle. AAAShiroFilter enables
authentication by default.

Change-Id: I1e0e36d1240c52ca950c4370e27bd0edd6ddfca1
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoProvide support for new grafana and re-add dataquery feature 91/69691/1
AbdulRahman AlHamali [Tue, 20 Mar 2018 16:34:04 +0000 (18:34 +0200)]
Provide support for new grafana and re-add dataquery feature

Change-Id: I3f63e8deddf6b4f0fc61af226977eeaad23b4896
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoDo not pull inventory model in restconf/netflow-collector 56/69456/2
Robert Varga [Tue, 13 Mar 2018 18:55:13 +0000 (19:55 +0100)]
Do not pull inventory model in restconf/netflow-collector

Thes artifacts is not referencing the model in any discernible
way, do not pull it in to clean up feature references.

Change-Id: Icc40ed68af124bd160ee3e6634fa6c186fd2aad3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
6 years agoBump versions by x.(y+1).z for next dev cycle 54/68754/1
Anil Belur [Mon, 26 Feb 2018 07:41:47 +0000 (17:41 +1000)]
Bump versions by x.(y+1).z for next dev cycle

Change-Id: I3fe501b823d19a9d7df8fb2bd6ed569fcfbfd0fd
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoMerge "Add DynamicImport-Package to hbase bundle"
Tom Pantelis [Thu, 15 Feb 2018 14:37:47 +0000 (14:37 +0000)]
Merge "Add DynamicImport-Package to hbase bundle"

6 years agoAdd DynamicImport-Package to hbase bundle 03/68303/1
Tom Pantelis [Thu, 15 Feb 2018 13:26:16 +0000 (08:26 -0500)]
Add DynamicImport-Package to hbase bundle

Alleviates ClassNotFoundEx due to dynamic Class.forName calls
in the hbase lib.

Change-Id: Id7822a39f24de2f00c5da7266c03d54aadc18e33
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoAdd karaf distro project 64/68164/6
Tom Pantelis [Mon, 12 Feb 2018 23:38:21 +0000 (18:38 -0500)]
Add karaf distro project

Change-Id: I9d1d6b726c500b6b2fbd6755eb63c9244cb81654
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoMerge "Remove restconf feature from odl-tsdr-core"
Tom Pantelis [Thu, 15 Feb 2018 12:52:37 +0000 (12:52 +0000)]
Merge "Remove restconf feature from odl-tsdr-core"

6 years agoRemove restconf feature from odl-tsdr-core 79/68279/1
Tom Pantelis [Thu, 15 Feb 2018 03:54:54 +0000 (22:54 -0500)]
Remove restconf feature from odl-tsdr-core

SFT's are failing on jenkins due to karaf JVM crashing
apparently when attempting to restart bundles due to the
restconf feature.

Change-Id: I824b5a75feb5b262e9ea064cd20d0852f38222fd
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoMerge "Use openflowplugin instead of l2switch. And remove unneeded dependence in...
Tom Pantelis [Thu, 15 Feb 2018 03:49:07 +0000 (03:49 +0000)]
Merge "Use openflowplugin instead of l2switch. And remove unneeded dependence in netflow collector"

6 years agoUse openflowplugin instead of l2switch. And remove unneeded dependence in netflow... 13/68013/1
AbdulRahman AlHamali [Wed, 7 Feb 2018 12:20:16 +0000 (14:20 +0200)]
Use openflowplugin instead of l2switch. And remove unneeded dependence in netflow collector

Change-Id: I7411e46d26306ba8ac9d118aa0581d652ddaa7b3
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoMigrate command completion to karaf4 84/67984/2
AbdulRahman AlHamali [Tue, 6 Feb 2018 21:37:42 +0000 (23:37 +0200)]
Migrate command completion to karaf4

Change-Id: Ibec0c840fb8522f1fc45470fbde19b9dcc9f3030
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoUncomment odl-tsdr-syslog-collector feature 39/67339/2
AbdulRahman AlHamali [Fri, 19 Jan 2018 08:04:51 +0000 (10:04 +0200)]
Uncomment odl-tsdr-syslog-collector feature

Change-Id: I13f06df6a4e49888e129617aaafdd72a2ec9e276
Signed-off-by: AbdulRahman AlHamali <a.alhamali93@gmail.com>
6 years agoBump to yangtools-2.0.0 and odlparent-3.0.2 28/66528/7
Tom Pantelis [Fri, 15 Dec 2017 20:22:51 +0000 (15:22 -0500)]
Bump to yangtools-2.0.0 and odlparent-3.0.2

Also upgrades jline version to 3.5.0, to be aligned with karaf.

Change-Id: I2e244fd1cf8fd6056813ba8626a049a44de6f24e
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
Signed-off-by: Robert Varga <rovarga@cisco.com>
6 years agoFix odlparent 3 Checkstyle issues 27/66527/2
Tom Pantelis [Fri, 15 Dec 2017 23:25:50 +0000 (18:25 -0500)]
Fix odlparent 3 Checkstyle issues

Change-Id: I87e62d5ec83cfa139d3bebf0f5067368fc0a04f3
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
6 years agoConvert remaining bundles to use blueprint annotatioons 24/66324/3
Tom Pantelis [Fri, 8 Dec 2017 20:29:42 +0000 (15:29 -0500)]
Convert remaining bundles to use blueprint annotatioons

Change-Id: I66d33649932f2f58858c52af325df1350ff7894b
Signed-off-by: Tom Pantelis <tompantelis@gmail.com>