Ivan Hrasko [Tue, 12 Nov 2024 08:26:41 +0000 (09:26 +0100)]
Use /openapi/api/v3/mounts without ending slash
New Netty based RESTCONF OpenApi server does not recognize
URL when it ends by slash ('/').
Adapt swagger-initializer accordingly.
JIRA: NETCONF-1370
Change-Id: Ie585e778686fff84eb091847ac9cbad63cacdd69
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Ruslan Kashapov [Thu, 5 Sep 2024 07:06:43 +0000 (10:06 +0300)]
Netty endpoint for RESTCONF OpenApi
Introduce OpenApiResourceProvider, which registers to each
NettyEndpoint's WebHostResourceProvider whiteboard.
Each NettyEndpoint then drives instantiation of its own
WebHostResourceInstance for each registered provider.
JIRA: NETCONF-1370
Change-Id: I4005f3bd1fe238672cfdee465474d625a628a561
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Robert Varga [Sun, 10 Nov 2024 19:16:25 +0000 (20:16 +0100)]
Split out ServerRequestExecutor
This is a reusable piece, not directly tied to the pipeline. Split it
out, making for a leader HTTPServerSession.
JIRA: NETCONF-1424
Change-Id: Ifc9e79ec32d91a2871a65e8b751267dec0cfbe67
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sun, 10 Nov 2024 18:35:55 +0000 (19:35 +0100)]
Shutdown threadpools when channelInactive()
A secodary event that should trigger us shutting down is when the
channel does down. Add that hook.
JIRA: NETCONF-1424
Change-Id: I14390525996a11da12feaffa3802db2e7be13d73
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 12 Nov 2024 03:27:41 +0000 (04:27 +0100)]
Make sure RestconfStream.Sender gets non-null
null event chunks are simply not allowed, annotate as much.
Change-Id: I9e97cc5ef1756106324d440b3bffca392561d1f6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Yaroslav Lastivka [Fri, 8 Nov 2024 13:52:44 +0000 (15:52 +0200)]
Add credentials for CallHome Netconf node
Added empty unencrypted login credentials configuration for
CallHome Netconf nodes in CallHomeMountService.
Ensures compatibility with NetconfDeviceTopologyAdapter,
which now requires credentials for creating NetconfNode
after applying NETCONF-1235.
JIRA: NETCONF-1427
Change-Id: I24b1020ca10229bb394e3a2c8930f8269dd86d4e
Signed-off-by: Yaroslav Lastivka <yaroslav.lastivka@pantheon.tech>
Robert Varga [Mon, 11 Nov 2024 12:59:09 +0000 (13:59 +0100)]
Modernize callhome-provider
Use DataObject{Identifier,Reference} instead of InstanceIdentifier.
Change-Id: I9f976c71372e3b822e4fe7cc00450dcdf9860082
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Mon, 21 Oct 2024 09:22:59 +0000 (11:22 +0200)]
Indicate RESTCONF server features
Indicate that RESTCONF server implementation supports:
HttpListen and HttpsListen features.
HttpsCallHome will be supported after server refactoring
is completed and we will have a user for it.
CentralRestconfServerSupported feature is going to be relocated
into new apps/central-restconf-server package.
Change-Id: I0a52ed7918547ef39f18e1dcfc0f473cc5a4f857
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Peter Suna [Thu, 7 Nov 2024 12:24:14 +0000 (13:24 +0100)]
Stop device reconnection with wrong config
Call onDeviceFailure directly on delegate attribute instead
of calling onDeviceFailed method, which tries to reconnect with
same incorrect configuration.
JIRA: NETCONF-1425
Change-Id: I0b5e09fadcca0e140d16d01f94abdf9cd882e8a1
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Robert Varga [Wed, 6 Nov 2024 23:41:45 +0000 (00:41 +0100)]
Fix BytebufRequestResponse capitalization
The response holds a ByteBuf, not a Bytebuf. Capitalize correctly.
Change-Id: Ibb6fcce0794b83af6e8ab13c026342d770288d83
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 30 Oct 2024 12:27:59 +0000 (13:27 +0100)]
Use virtual threads for request execution
HTTPServerSession has four distinct phases, two of which involve
reading or writing message body.
This patch offloads processing from the Netty thread, so that the task
of parsing request body and wrapping of asynchronous execution occurs
on a dedicated virtual thread -- off-loading IO threads and allowing
Channel.write() to occur in a blocking context.
JIRA: NETCONF-1424
Change-Id: Ia347e673d824ca18374a390284095410b361c5a4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 7 Nov 2024 00:06:09 +0000 (01:06 +0100)]
Refactor RestconfSession.{close,registerResource}
There is no need to have an explicit listener on Channel.closeFuture(),
as we can just attach to channelInactive().
This patch also refactor resource registration, as RPC invocations,
which are expected to be the primary source of interactions, are
expected to execute asynchronously.
JIRA: NETCONF-1424
Change-Id: I8c257f8cfd427e00b066651c463298c708782eca
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 30 Oct 2024 12:03:31 +0000 (13:03 +0100)]
Use channel allocator for Responses
Refactor HTTPServerSession to correctly take advantage of ReadyResponse
and of channel's allocator -- leading to potential buffer reuse.
This also ensures we release the allocated buffer is released in case
of a writeout failure.
Change-Id: Ifb603aefee753fd9c0bc3c38f65e0eaa3cff2a6f
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 6 Nov 2024 11:14:22 +0000 (12:14 +0100)]
Add ExceptionRequestResponse
This is a specialized class for reporting error responses with a known
exception cause.
Change-Id: I1bb6e480d971bc1f9f2a4cd57aef61ff8eca2932
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sun, 3 Nov 2024 11:44:01 +0000 (12:44 +0100)]
Recognize */* media range
While our processing is quite lacking, add a special-case to allow for
*/* being specified.
Change-Id: I7b0cee7ad03185f8be07db4ae7394cc3448d605d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 22 Oct 2024 15:48:09 +0000 (17:48 +0200)]
Add WebHostResource
We need the ability to inject secondary services into the NettyEndpoint,
such that they have a reference to the XRD (for now).
This patch introduces WebHostResource as the common execution API, which
are dispatched to from EndpointRoot.
This maps neatly to OSGi whiteboard pattern:
- WebHostResourceProvider is the service published to the whiteboard
- OSGiNettyEndpoint instances are event sources
In other environments SimpleNettyEndpoint's explicit register()/close()
mechanics might prove to be helpful
JIRA: NETCONF-1370
Change-Id: I041059919db7471a11c5e0da75557ff3c48efcda
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 30 Oct 2024 11:59:17 +0000 (12:59 +0100)]
Fix ReadyResponse methods
ReadyResponse implementations should not need an allocator, make sure to
adjust methods shapes to have subclasses reflect that.
Change-Id: Iba8f9f574a2990d6e8c36e541f71d0b6e06ad19c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 7 Oct 2024 08:38:01 +0000 (10:38 +0200)]
Add Automatic-Module-Name to transport-{http,ssh}
We need to start using these in JPMS modules, let's make sure their
module names are stable.
transport-ssh needs the entire 'shade sshd' story figured out: we
essentially want to have a module which includes sshd, but does not
export it.
transport-http uses codegen and that cannot be modules just yet.
Change-Id: Ic71d637b9cf370ea73408b15f6a8fbff24bb3f33
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Peter Suna [Fri, 25 Oct 2024 12:23:18 +0000 (14:23 +0200)]
Don't update data if DeviceSalFacade was closed
Verify that NetconfTopologyDeviceSalFacade and NetconfDeviceSalFacade
are not closed before modifying device data. This can prevent
the device from connecting if one thread calls close method while
another thread calls onDeviceConnected method.
JIRA: NETCONF-1418
Change-Id: Ifbe9a46a2f4ac236b63d9999fdaf84b9ec7ec563
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Samuel Schneider [Wed, 16 Oct 2024 10:39:40 +0000 (12:39 +0200)]
Improve event notification subscription tutorial
Information about possible query parameters was misplaced.
Query parameters should be used when connecting to SSE location
not when obtaining it.
Move this information to more relevant place.
JIRA: NETCONF-1392
Change-Id: Idb44f76ea1fa5a894b567b025b8871425efe4436
Signed-off-by: Samuel Schneider <samuel.schneider@pantheon.tech>
Robert Varga [Thu, 26 Sep 2024 12:03:09 +0000 (14:03 +0200)]
Update public TransportChannelListener references
Fix raw type warnings stemming from TransportChannelListener being
generic in public interfaces. This constitutes an API-incompatible
change.
JIRA: NETCONF-1401
Change-Id: I87097ef38fddce12afa0b667a16e3ffe66994ef2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 24 Oct 2024 13:52:58 +0000 (15:52 +0200)]
Bump versions to 9.0.0-SNAPSHOT
This starts the next development iteration.
Change-Id: I2bb7447d2269b88a5ecc49c7cdbb1b880a1e3087
Signed-off-by: Yaroslav Lastivka <yaroslav.lastivka@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 24 Oct 2024 13:51:37 +0000 (15:51 +0200)]
Bump versions to 8.0.4-SNAPSHOT
This starts the next development iteration.
Change-Id: I972d7d73489e2dcff1b8dbd865503921bbdb4f86
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
jenkins-releng [Thu, 24 Oct 2024 11:12:09 +0000 (11:12 +0000)]
Release netconf
Robert Varga [Thu, 24 Oct 2024 09:27:08 +0000 (11:27 +0200)]
Propagate HttpHeaders to WellKnownResources
We will need to see the headers to deal with the Accept-based JRD
serving. This patch exposes that.
JIRA: NETCONF-1379
Change-Id: Ic3551be618eda818365bb8a974dd53adf3de79ab
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Matej Sramcik [Tue, 10 Sep 2024 14:34:25 +0000 (16:34 +0200)]
Separate transport from service layer
This patch extracts all JAX-RS-specific contracts from OpenApiService
and moves them to a new class, JaxRsOpenApi.
We then use WebInitializer to instantiate JaxRsOpenApi based on injected
OpenApiService instance.
JIRA: NETCONF-1370
Change-Id: Ifff963e4409e8e3c23add65beb88c62111ae24a8
Signed-off-by: Matej Sramcik <matej.sramcik@pantheon.tech>
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Wed, 9 Oct 2024 15:05:22 +0000 (17:05 +0200)]
Register resources for cleanup
Add the possibility to close transport session and the ability
to register resource to be closed.
For this we also add onTransportChannelClosed() into
TransportChannelListener which is invoked when channel is closed.
JIRA: NETCONF-714
Change-Id: I4912bd7d4a03530e4941bc9b663b2dc3b54a2f22
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Signed-off-by: Samuel Schneider <samuel.schneider@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 10 Oct 2024 14:29:22 +0000 (16:29 +0200)]
Force instanceNum to be a long
We require this parameter and will issue a 500 internal server error if
it is not present.
Push the definition to be a plain long, so as to guide JAX-RS (and
future users) towards non-Strings.
JIRA: NETCONF-1370
Change-Id: Ia01b09721536a1a343ab27beff093fb8f107856a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 23 Oct 2024 14:02:43 +0000 (16:02 +0200)]
Move NO_CONTENT and NOT_FOUND response
These responses are rather useful, make them more widely available.
JIRA: NETCONF-1379
Change-Id: I35c76c89afeb05458337a268d1833524a2e89cc5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 22 Oct 2024 20:54:59 +0000 (22:54 +0200)]
HTTPServerSession.prepare() is version-agnostic
HttpVersion is only required for HttpResponse construction, i.e. we have
it passed to Response.toHttpResponse(HttpVersion). Remove them from
prepation path.
Change-Id: I0aeadf8a5e97ef9ffe4d98ab6a1405173a18bc83
JIRA: NETCONF-1379
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 22 Oct 2024 10:38:54 +0000 (12:38 +0200)]
Mark outbound write thread is internal
When we are writing out a message, we may be hitting a KEX -- at which
point we would normally be blocked.
The problem is that the blocked thread is usually the Netty IO thread,
which effectively ends up deadlocking it.
Prevent this situation by marking our thread as internal, which causes
SSHD to enqueue the message rather than blocking.
JIRA: NETCONF-1415
Change-Id: If472986270e827a316099168069754fe1661939d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Tue, 22 Oct 2024 11:00:44 +0000 (13:00 +0200)]
Fix javadocs in Subscriber(s)
Fix typos and wrong links in Subscriber and Subscribers javadocs.
Make clear that we operate on RestconfStream.
Change-Id: I16b3388a3e5bdd74048f1c7993f3d0dbc0b46ad8
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Robert Varga [Tue, 22 Oct 2024 10:41:42 +0000 (12:41 +0200)]
Bump upstreams
Adopt:
- mdsal-14.0.4
- controller-10.0.4
- aaa-0.20.3
Change-Id: I4317628af4a0b24004919d3faee45f955c423ca2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Mon, 21 Oct 2024 08:52:36 +0000 (10:52 +0200)]
Fix docs in RestconfStream
We are no more using web-sockets but SSE.
Fix documentation accordingly.
Change-Id: I656e5c77071413479c2ce62e83113f0bf62ae584
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Ivan Hrasko [Mon, 21 Oct 2024 08:51:13 +0000 (10:51 +0200)]
Fix docs in SSESender
We are no more using web-sockets but SSE.
Fix documentation accordingly.
Change-Id: Id1aab876a7c63ee2d31e3890775645c6888e9e46
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Peter Suna [Wed, 9 Oct 2024 10:48:43 +0000 (12:48 +0200)]
Add range restriction for backoff-multiplier
Add range restriction from 1 to max value. This will remove
the case when it starts decreasing instead of multiplying the value
initially set as the minimum value.
JIRA: NETCONF-1407
Change-Id: I2bb906386b9935543e8a4b6eab23e90313d3f4f2
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Robert Varga [Sun, 20 Oct 2024 08:26:11 +0000 (10:26 +0200)]
Clean up XRD output
Use writeEmptyElement() to ensure we have a self-closed tag.
JIRA: NETCONF-1379
Change-Id: I2112e42b3cfccbe683bd9f618f9fd072825b5410
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 17 Oct 2024 16:49:15 +0000 (18:49 +0200)]
Share a single DOMSchemaService across devices
Since we have the same model context, we can just use a
FixedDOMSchemaService, which does the magic.
This reduces the amount of code we have as well as eliminate a single
object per instance, improving scalability a bit.
Change-Id: I28e0f468647b54b49623065eabd816b326474dd9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 17 Oct 2024 16:19:12 +0000 (18:19 +0200)]
Add FIXME to MdsalOperationService
JIRA: NETCONF-1414
Change-Id: I9e1587bafabb5e93d340e8e079e95223df764101
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 21:36:18 +0000 (23:36 +0200)]
Introduce transport.spi
We have a number of implementation-internal classes living in
transport.api. Create transport.spi, so that implementation details are
in a different package.
Change-Id: I4701c632dc41f8202214068495711c640acd49a4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 16 Oct 2024 10:49:03 +0000 (12:49 +0200)]
Handle the case of a null private key
A null object is an indication that there are no more objects left --
handle that gracefully.
A null private key can cause an exception during the creation of a
PEMParser instance. A check has been added to ensure that the
private key is not null.
JIRA: NETCONF-1411
Change-Id: Iebe4da892bfcc518882403b838899060b4deea6b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Signed-off-by: Yaroslav Lastivka <yaroslav.lastivka@pantheon.tech>
Peter Suna [Tue, 15 Oct 2024 08:26:20 +0000 (10:26 +0200)]
Clarify description in odl-netconf-device
Update description for max-backoff-millis and min-backoff-millis
leaf to clarify their functionality.
JIRA: NETCONF-1408
Change-Id: Iccefcad853c49303402fda21c01748cb21172539
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Ivan Hrasko [Wed, 16 Oct 2024 11:05:27 +0000 (13:05 +0200)]
OpenApi: Fix /mounts endpoint
After removal of JacksonJaxbJsonProvider usage there is no writer
capable to write List of MountPointInstance(s) to JSON.
Create dedicated MountPointsEntity to generate OpenApiEntity.
JIRA: NETCONF-1412
Change-Id: I6f645deacc51841d00d0fd2db574c26163ca18f2
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 03:58:01 +0000 (05:58 +0200)]
Modernize keystore-legacy
Use DataObjectIdentifier instead of InstanceIdentifier.
Change-Id: I030c8dfc89fa3d1707c8db54aa8d7777ee12ff06
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Wed, 16 Oct 2024 07:44:46 +0000 (09:44 +0200)]
Address NETCONF-1210 FIXME
NETCONF-1210 has been fixed thus we can un-comment assertion.
Change-Id: I5c98363a6e5fd0a5c3cd97d2f8abddf1a89de54e
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Oleksandr Zharov [Fri, 9 Aug 2024 16:42:32 +0000 (18:42 +0200)]
OpenAPI: remove empty list/container from payloads
Adjusted depth logic to remove empty lists and containers.
Now logic should work in line with proposed in NETCONF-1298.
Removed POST for depth 1 - for that value child container/list
always result in empty payload because child elements are being
cut off by depth.
JIRA: NETCONF-1345
Change-Id: I4f83823ac06d63dc57c69963f0eb8f6a1e3bb671
Signed-off-by: Oleksandr Zharov <oleksandr.zharov@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 19:04:07 +0000 (21:04 +0200)]
Fix client dispatcher queuing
Future listeners are not delivered immediately upon write completion,
but rather at some time after.
This means we need to setup the queue before we send the request, so
that if we receive a response before the callback runs we do not get
into weird state.
JIRA: NETCONF-1406
Change-Id: I66e1911fd6db6843499f772827e7ae00fa3ff377
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 15:01:37 +0000 (17:01 +0200)]
Move AbstractJukeboxTest
We have contracts in restconf.server.api which could use some localized
testing. Allow that to happen by rehosting AbstractJukeboxTest.
Change-Id: Ic31f667a3bee4f661f2f1b5555b25e24c7e72c7d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 10:59:29 +0000 (12:59 +0200)]
Adopt draft-ietf-netconf-restconf-client-server-38
Pick up latest upgrades to the draft.
Change-Id: Ife09d7ba979f52ad9e812b587c0af521a31498c0
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 19:57:11 +0000 (21:57 +0200)]
Eliminate TestUtils
This contains only a single mis-named method and a number of static
nested classes.
This patch:
- converts nested classes to top-level, reducing the need for imprts
- moves freePort() to AbtractE2ETest.randomBindablePort() with some
documentation
- introduces transport.http.Http{Client,Server}StackConfiguration
for wide reuse.
Change-Id: I7f269551654761545406e591c7e7562ea5aacf70
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 20:58:18 +0000 (22:58 +0200)]
Allocate TCPClient earlier
Mirror the layout of TCPServer, where we allocate the handle beforehand
-- which implies a validity check before we attempt to do anything.
Change-Id: Ib26512a3f36e4156a5c33ecccf2c7f8486648f78
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 20:43:40 +0000 (22:43 +0200)]
Use Empty.immediateFuture()
yang.common.Empty has a utility method, use that instead of our own
constant.
Change-Id: I6e2f59367c066467ad6df8d6ba285633e810157a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 18:44:05 +0000 (20:44 +0200)]
Introduce transport.api.NettyTransportSupport
Rehost most of the logic from transport-tcp to transport-api, so it can
be used by both transport-tcp and transport-udp.
Change-Id: Ie52a6365a9aae28f9509eeab6be260db3ac97be8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 15:46:45 +0000 (17:46 +0200)]
Expand NettyImpl with datagram support
Add basics needed to support datagram channels. This will be useful for
transport-udp.
Change-Id: I290f7677cf581fc66ccffbaf50dfc942388f1e12
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 16:10:27 +0000 (18:10 +0200)]
Disconnect AbstractNettyImpl from ietf-tcp-common
The Netty implementation should not have direct bindings to a concrete
model, so it can be reused between transport-{tcp,udp}.
Change-Id: I447163b7db2d2c1bfccae9042c661e3c2d999bfb
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 17:53:18 +0000 (19:53 +0200)]
Fix NettyTransportSupport documentation
We have a copy&paste error in javadoc links, fix that up.
Change-Id: Ib0171370c7806d5012b7788b294e7bb38fec06e4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Ivan Hrasko [Wed, 4 Sep 2024 11:30:25 +0000 (14:30 +0300)]
End-to-end tests for Netty RESTCONF server
Integration tests provided for Netty based RESTCONF server
implementation. Instance of Netty endpoint with basic
controller models is created then HTTP requests are performed
using Netty based client from transport-http.
Tests covering requests to /data (incl actions and yang-patch),
/operations, /yang-library-version, /modules URIs plus
/.well-known/host-meta{.json} URIs.
Additional scenarios:
- topology node mount using device simulator
- data change notifications event stream
- device notification event stream
JIRA: NETCONF-1367
Change-Id: Ibb9dc960cc03b1a7215cb8ff39e2fd2bd136db0e
Signed-off-by: Ivan Hrasko <ivan.hrasko@pantheon.tech>
Signed-off-by: Oleksandr Zharov <oleksandr.zharov@pantheon.tech>
Signed-off-by: Matej Sramcik <matej.sramcik@pantheon.tech>
Signed-off-by: Ruslan Kashapov <ruslan.kashapov@pantheon.tech>
Signed-off-by: Yaroslav Lastivka <yaroslav.lastivka@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 13:53:30 +0000 (15:53 +0200)]
Clean up ClientFactoryManager configuration
We have split configuration processing -- the key is acquired only after
the connection is established. Refactor this by acquiring the bits
needed and then perform minimal changes to ClientFactoryManager.
This also adds a FIXME for a follow up, which would not require access
to SSH implementation internals.
JIRA: NETCONF-1190
Change-Id: I223597f3b2cfc7f5e898e830874d3e142eb10acc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 14 Oct 2024 11:59:22 +0000 (13:59 +0200)]
Convert transport-ssh to a JPMS module
Now that we are not hosting models, we can become a full JPMS module. As
a temporary side-effect, also specify Automatic-Module-Name in
shaded-sshd.
JIRA: NETCONF-1190
Change-Id: Ia19cec291d368dcfe75a97716003ba633685e10b
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Yaroslav Lastivka [Thu, 10 Oct 2024 10:19:10 +0000 (13:19 +0300)]
Add encoding to AbstractPendingRequest
Fix picking encoding inside AbstractPendingRequest#onFailure
which causes NPE during the initialization of FormattableDataResponse.
JIRA: NETCONF-1379
Change-Id: I336a28b20f08402f79636dc8a97c4e15d1282d01
Signed-off-by: Yaroslav Lastivka <yaroslav.lastivka@pantheon.tech>
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Peter Suna [Wed, 9 Oct 2024 11:11:36 +0000 (13:11 +0200)]
Ensure backoffMillis is not less than expected
Keeping the last backoff value after randomization can
reduce the SleepingTask timeout to a much smaller value
than expected.
Fix this by keeping the backoff value before randomization.
JIRA: NETCONF-1408
Change-Id: Ie926c92f9e203515c9623d9c794e441a447962f7
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 10:53:32 +0000 (12:53 +0200)]
Adopt draft-ietf-netconf-udp-client-server-04
Import ietf-udp-{client,server}.yang as found in the latest version of
draft-ietf-netconf-udp-client-server -- in transport-udp.
Change-Id: Id483a3f195346ebea51530be35cd6cb0d4c321e9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 10:44:16 +0000 (12:44 +0200)]
Adopt draft-ietf-netconf-netconf-client-server-37
Upgrade models to match current draft, at revision 2024-08-14.
Change-Id: Iee29808e943abf4945d7d11876f9ce57af5e420d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Tue, 15 Oct 2024 09:28:02 +0000 (11:28 +0200)]
Fix odl-netconf-transport dependencies
Make sure we use a range for odl-mdsal-model-rfc7317-iana-crypt-hash.
Change-Id: I555ea90c0c840ee3ee107687d4e32336028571ea
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
(cherry picked from commit
325167b5cb205456cbc6a5fe8fd84606c67d1c1c)
Robert Varga [Tue, 15 Oct 2024 10:29:33 +0000 (12:29 +0200)]
Convert transport-tls to a JPMS module
Now that we do not have generated code here, we can switch to being a
JPMS module, properly encapsulating our internals.
Change-Id: Ib8c06203408f53b40db996b868329ae928dd1745
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 21:00:29 +0000 (23:00 +0200)]
Switch to using RFC9645 models
ietf-tls-{client,common,server}.yang have been published in RFC9645 and
are available from MD-SAL.
Change-Id: If36b3c6070223fc81c34354bbcc6c4ea96289be2
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Sangwook Ha [Mon, 14 Oct 2024 17:45:31 +0000 (10:45 -0700)]
Allow status code 201
The POST request returns 201 when resource is created successfully,
so it should not generate WARN log when the test tool sees the response
code. Include it in the list of accepted status codes.
JIRA: NETCONF-1410
Change-Id: I6b1d5e099bba285ecf6f1653ef9ce188eb12bd8c
Signed-off-by: Sangwook Ha <sangwook.ha@verizon.com>
Robert Varga [Mon, 14 Oct 2024 12:12:48 +0000 (14:12 +0200)]
Refactor SessionListener integration
Disconnect SSHTransportStack from SessionListener, so the implementation
detail is hidden from users. An additional side-effect is that we no
longer need to suppress SpotBugs.
JIRA: NETCONF-1190
Change-Id: I79c28aba49064289dfa7ce903522b1da60482004
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 14 Oct 2024 10:53:48 +0000 (12:53 +0200)]
Convert transport-tcp to a JPMS module
We have moved models out, convert now to a JPMS module to improve Java
integration.
Change-Id: I3eb8603af7a66f1f102d9a126bbdad492ee334cc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 19:50:55 +0000 (21:50 +0200)]
Switch to using RFC9644 models
ietf-ssh-{client,common,server}.yang have been published in RFC9644 and
are available from MD-SAL.
This patch switches to using them instead of the old draft, adapting to
the new way how algorithms are modeled.
Change-Id: Iaac864c7ec315794c92f6bd8313a580a9d4b75ec
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 18:36:55 +0000 (20:36 +0200)]
Switch to using RFC9643 models
ietf-tcp-{client,common,server}.yang have been published in RFC9643 and
are available from MD-SAL.
This patch switches to using them instead of the old draft, but only
supports single address binding for now.
It also requires updating the client/server models to match the
multi-listen capability -- which we pick up in the exact shape as the
latests drafts.
Change-Id: Id2ee382b6d988679a65a2812b51af8653e22d6c1
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 18:32:11 +0000 (20:32 +0200)]
Use RFC9642 ietf-keystore.yang
This model has been finalized and published and is available from
MD-SAL. Pick it up from there, leaving keystore-api a vacuous artifact
for now.
Change-Id: Ia99d3d1947fb343586d32f8ae6e420af5ffd6588
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 18:26:28 +0000 (20:26 +0200)]
Use RFC9641 ietf-truststore.yang
This model has been finalized and published and is available from
MD-SAL. Pick it up from there, leaving truststore-api a vacuous artifact
for now.
Change-Id: I105a25b91ed2f06acec60295f1af716f7e69285d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 18:08:27 +0000 (20:08 +0200)]
Use RFC9640 ietf-crypto-types
MD-SAL is providing ietf-crypto-types.yang, hence we do not have to
package it ourselves anymore.
Change-Id: I066a1055f78882ba8c5ab26639f9654da0d4f95a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 16:28:40 +0000 (18:28 +0200)]
Bump upstreams
Adopt:
- odlparent-14.0.4
- infrautils-7.0.4
- yangtools-14.0.5
- mdsal-14.0.3
- controller-10.0.3
- aaa-0.20.2
Change-Id: Ide732539861eda8d123f66a2c5849f326f4ed2f6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 12 Oct 2024 17:25:01 +0000 (19:25 +0200)]
Fixup checkstyle
Upgraded checstyle is catching a lot of javadoc violations, this patch
fixes them up.
Change-Id: Ib156cb891e748b957cf0dd9415d043c35c62d01c
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 10 Oct 2024 08:46:39 +0000 (10:46 +0200)]
Add RFC6415 constructs
The XRD and related contracts should be modeled, so that we can actually
look them up.
This creates LinkRelation and WellKnownURI constant registries and
models the XRD, at least the very bare bones we require.
We then proceed to define responses and use them in WellKnownResources.
JIRA: NETCONF-1379
Change-Id: Id24fe3dc9d1798440277286d182506feabafdc55
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Thu, 10 Oct 2024 13:53:23 +0000 (15:53 +0200)]
Fix JAX-RS JRD output
'links' is an array, not a singleton -- correct that.
Change-Id: I10ef1929d1ec13a5a111e7063baeca62141892e8
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Peter Suna [Wed, 4 Sep 2024 13:51:15 +0000 (15:51 +0200)]
Remove temp directory usage in plaintext-localfile
The PlaintextLocalFileStorageTest use the /tmp folder to store
new data-file and then replace original data-file in custom location.
This can cause error when move operation is performed accross
data stores. Fix this issue by storing new data-file in same
folder as original data-file.
Also removed the usage of /tmp folder in the tests.
JIRA: NETCONF-1396
Change-Id: I3124253567d9d47e59c61aa49f193980cff15d69
Signed-off-by: Peter Suna <peter.suna@pantheon.tech>
Robert Varga [Thu, 10 Oct 2024 10:29:01 +0000 (12:29 +0200)]
Introduce transport.http.ReadyResponse
ReadyResponse is a Response which can seamlessly translate to
HttpResponse, without throwing a checked exception.
Also promote EmptyRequestResponse to transport.http.
JIRA: NETCONF-1379
Change-Id: I8ce712a15dd692dcdd03cf00c9f92ed0be80a0d5
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 06:20:04 +0000 (08:20 +0200)]
Modernize netconf-server-mdsal
Eliminate most of deprecation warnings by migrating to
DataObjectIdentifier.
Change-Id: I6fb2a22be4dc7c31d8f04907fa5da79fef71802a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 12:26:07 +0000 (14:26 +0200)]
Use switch expression to assign type
Address the TODO we have left here.
Change-Id: I83191cd97b8fd2c6325325866f5f61b1483e2156
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 12:01:35 +0000 (14:01 +0200)]
Reduce argument passing in PathsEntity
We are passing down arguments from fields. Stop doing that and reference
the fields instead.
Change-Id: I6e3d4143707426b81a4219e11e695a742c9d497d
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 11:08:33 +0000 (13:08 +0200)]
Add OpenApiServiceImpl.unboxOrZero()
Using Objects.requireNonNullElse() operates on objects, hence we are
bouncing between Integer and int.
Add unboxOrZero(), which does what it says on the tin: unboxes non-null
and returns 0 for nulls.
Change-Id: Ic69b4284dc0ddcabf69e46d7e458970e5b771dc9
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 19:35:26 +0000 (21:35 +0200)]
Refactor PreparedRequest/Response class hierarchy
PreparedRequest, along with its sibling API components should really
live in transport-http.
This patch pulls it down, clreaning up the class hierchy.
JIRA: NETCONF-1379
Change-Id: Idada210d5236675037424650a52546bf6155c6a3
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 5 Oct 2024 06:25:26 +0000 (08:25 +0200)]
Fix SSH client re-authentication
The onKeyEstablished() callback is invoked whenever a key is
established, i.e. also after rekeying.
This leads to us attempting to trigger authentication again, which is
stopped cold by an ISE.
Fix this by bringing back the check we used to have in
NetconfCallHomeServer.
JIRA: NETCONF-1405
Change-Id: I2a15c319473f3a7a1082f5f58a9a30e9fbec8d73
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 07:19:03 +0000 (09:19 +0200)]
Modernize MonitoringToMdsalWriter
Switch to using DataObjectIdentifier.
Change-Id: Ib4ebe27d2bd90fd056eac627f72c153e50f35c94
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 09:00:46 +0000 (11:00 +0200)]
Eliminate OpenApi streams
There is no need to perform complicated buffering, as we have an
OpenApiEntityWriter.
We perform the following refactors:
- OpenApiInputStream is now DocumentEntity
- ComponentsStream is (for) now ComponentsEntity
- SchemasStream is now SchemasEntity
We also instantiate primary entities eagerly, as now they are costing us
nigh nothing and their presence makes it clearer as to what state is
encapsulated where.
JIRA: NETCONF-1370
Change-Id: I235ef442fddae47591348c2e73419972d3265c03
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 08:50:01 +0000 (10:50 +0200)]
Fix invalid use of @NonNull
Primitive types are not references, hence they can never be null.
Change-Id: Ia68c25593c3a4383970d59f5588f5ce6ba9cf2bc
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 06:25:04 +0000 (08:25 +0200)]
Migrate to BindingDataCodecFactory
BindingDOMCodecFactory is deprecated, use its replacement.
Change-Id: I224714448be903660b635426f6b675007d6bbf34
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Wed, 9 Oct 2024 03:45:35 +0000 (05:45 +0200)]
Modernize yanglib-mdsal-writer
Use DataObjectIdentifier instance of InstanceIdentifier.
Change-Id: I9b4ce5f765d2a28ee0cad45cdd2e38f3a5e6ff65
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 10 Aug 2024 01:11:25 +0000 (03:11 +0200)]
Add ChunkOutputStream
Address the TODO left by the previous patch: ChunkOutputStream knows
the output and chunkSize, plus it gets a buffer.
write() methods now append to the buffer and frame it to output as
needed.
While we are here, also improve toString() implementations and add more
documentation.
Change-Id: I5cf0b91ab593bd9ab9151051bf828c9877f1386e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 5 Oct 2024 12:24:26 +0000 (14:24 +0200)]
Move EndpointInvariants
EndpointInvariants is a simple DTO, we can just move it to
impl package without exposing anything to the outside world.
JIRA: NETCONF-1379
Change-Id: I282a908b771132bcb71b5098c87f23b3f3b62fa4
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Mon, 7 Oct 2024 07:51:56 +0000 (09:51 +0200)]
Move SegmentPeeler
SegmentPeeler is a utility useful to all users to transport-http, move
it there.
JIRA: NETCONF-1379
Change-Id: I281d48c548cee2752c58f13ec58320a6fddb8b8e
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Sat, 5 Oct 2024 12:16:16 +0000 (14:16 +0200)]
Finish RestconfSession/PendingRequest integration
We have all the bits and pieces concentrated in RestconfSession, now
implement request tracking as outlined in the FIXMEs, completing the
dispatch integration.
JIRA: NETCONF-1379
Change-Id: Icc5216b5496abda90cd2f1368f99134a07b09306
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 4 Oct 2024 20:53:31 +0000 (22:53 +0200)]
Split out HTTPServerSession
RestconfSession has a top-half, which is RESTCONF-specific, and a
bottom-half, which is closely related to HTTPServer's channel setup.
This splits out HTTPServerSession, so that it can be moved to
transport-http and reused by other components if/when the need arises.
JIRA: NETCONF-1379
Change-Id: Iab96baab4b9469a6a5eaf2d3234cfdbb813a59a6
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 4 Oct 2024 23:03:21 +0000 (01:03 +0200)]
Eliminate Http2Utils
This class is a case of false sharing:
1. most of the callers of copyStreamId() are server-side, so they
can squat in ServerSseHandler.
2. connectionHandler() is differentiated by a boolean -- which selects
the framer and deals with the setup of listeners.
Eliminate the entire class at the cost of duplicating ~15 lines of code.
While that may sound horrible, it is actually what we want to do,
because now we can evolve the setup on client side and server side of
things separately.
JIRA: NETCONF-1379
Change-Id: I3a5f4288f4b0d65fa53cc039e37b2dde70f46e87
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 4 Oct 2024 19:12:51 +0000 (21:12 +0200)]
Eliminate RestconfRequest
The sole remaining user of this class is RestconfSession, which uses
it just as an indirection.
Fold RestconfRequest.execute() into RestconfSession.executeRequest(), so
that we get all the things in one place.
JIRA: NETCONF-1379
Change-Id: I085d5c1e020d31ab1deb38c5496f1ace0bcd8ce7
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 4 Oct 2024 18:36:31 +0000 (20:36 +0200)]
Refactor AbstractRequestProcessorTest wiring
This patch finishes the 'RequestDispatcher' refactoring saga by wiring
AbstractRequestProcessorTest to RestconfSession so that all requests
we send end up going through all of our validation.
This flushes out the fact we do not add a Host: header, which this patch
adds to all testing messages.
JIRA: NETCONF-1379
Change-Id: I907d689e1524b501070d2ef96a00df77ad37aa25
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>
Robert Varga [Fri, 4 Oct 2024 18:11:17 +0000 (20:11 +0200)]
Improve PendingRequest.execute()'s body handling
Empty body is not really valid in terms of RESTCONF -- in fact we have
some code to detect empty inputs in restconf-server-api.
Update PendingRequest's contract to allow for null bodies and deal with
that in PendingRequest{With,Without}Body.
With that expressiveness in place, rework RestconfSession's content
reference handling, so that we release the body as soon as the execution
is done.
JIRA: NETCONF-1379
Change-Id: Idb3b92450cabe6fd27661e70863fa8b64e3bc93a
Signed-off-by: Robert Varga <robert.varga@pantheon.tech>