.. _controller_restconf: .. sectnum:: :start: 7 :depth: 3 ******** RESTCONF ******** This section describes how to use RESTCONF with the Clixon controller. Please also consult the RESTCONF section in the `Clixon user manual `_. Clixon provides two separate compile-time setups for RESTCONF: * `FCGI` / FastCGI: This solution uses a web reverse proxy such as NGINX. The reverse proxy configures all HTTP configuration. * `Native`: which combines a HTTP and Restconf server including openssl and nghttp2. Fcgi setup ========== If you use native mode, this section can be skipped. NGINX ----- Install NGINX. Edit the location as for example as the following minimal config:: location / { fastcgi_pass unix:/www-data/fastcgi_restconf.sock; include fastcgi_params; } On many NGINX installations this can be made in ``/etc/nginx/sites-available/default``. Restart NGINX:: sudo systemctl restart nginx.service Configure --------- You need to configure clixon for FCGI:: ./configure --with-restconf=fcgi NGINX typically creates a `www-data` user. The following must be done prior to running: * Create a directory for example at ``/www-data`` with the following permissions:: drwxr-xr-x 2 www-data www-data 4096 apr 1 20:14 www-data * Add the ``www-data`` user to the ``clicon`` group Ensure your controller.xml configuration file has the following entries:: clixon-restconf:allow-auth-none clixon-restconf:fcgi /usr/local/lib/controller/restconf Example config -------------- Next step is to setup RESTCONF in the datastore. FCGI configuration may look as follows:: true none /www-data/fastcgi_restconf.sock false 0 syslog Auth is set as none. If you want to use `basic auth`, you need to add support for authentication using the ``ca_auth`` plugin callback. You should modify the configuration above to suit you needs, thereafter install it in the Clixon datastore using one of the methods described in the next section. Fcgi install ------------ You install the RESTCONF configuration by adding it to the datastore in one of the following methods. Install using CLI ----------------- Enter the CLI and edit the RESTCONF configuration and commit it:: # clixon_cli cli> configure cli# set restconf enable true cli# set restconf auth-type none cli# set restconf fcgi-socket /www-data/fastcgi_restconf.sock cli# commit local cli# The commit command should (re)start the RESTCONF daemon with the new configuration. To verify that the RESTCONF is running, see Section `Verify the configuration`_. Install in datastore -------------------- If you use a `startup-db` or `running-db` you can directly edit the datastore by adding the restconf config and restart the ``clixon_backend``. Add the restconf config to the datastore, such as ``/usr/local/var/controller/startup.d/0.xml`` as follows:: ... true none /www-data/fastcgi_restconf.sock Then restart ``clixon_backend``, typically using systemd:: sudo systemctl restart clixon-controller.service Install using NETCONF --------------------- Send a NETCONF edit-config message to modify the restconf configuration, and then commit it:: urn:ietf:params:netconf:base:1.0 ]]>]]> true ... ]]>]]> ]]>]]> The RESTCONF daemon should be restarted automatically with the new configuration. Verify the configuration ------------------------ Verify that the daemon is running using the CLI:: cli> processes restconf status true running <--- You can also verify it via RESTCONF (fields are simplified):: POST /restconf/operations/clixon-lib:process-control HTTP/1.1 Content-Type: application/yang-data+json { "clixon-lib:input": { "name":"restconf", "operation":"status" } } A reply with a successful start is:: HTTP/1.1 200 { "clixon-lib:output": { "active": true, "description": "Clixon RESTCONF process", "status": "running" } } Native setup ============ Native mode is more complex to setup and provides many different configurations for RESTCONF. The controller supports the following: 1. Native TLS and http in the RESTCONF daemon. No reverse proxy is needed. 2. HTTP/1.1 and HTTP/2 3. Basic and TLS/SSL client cert authentication 4. Datastore configuration, not in configuration file Configuration ------------- You need to configure clixon for native:: ./configure --with-restconf=native Example config -------------- A typical RESTCONF native configuration may look as follows:: true client-certificate /etc/pki/tls/certs/clixon-server-crt.pem /etc/pki/tls/private/clixon-server-key.pem /etc/pki/tls/CA/clixon-ca-crt.pem default
192.168.32.1
443 true In the config where a TLS on port 443 on 192.168.32.1 is configured using client-certs placed in the ``etc/pki/tls`` directory. Alternatively, you may use `basic auth`, but then you need to add support for authentication using the ``ca_auth`` plugin callback. For testing purposes, ``none`` can be used as auth-type. Setup ===== You setup the connection to one or several devices by editing the device connection data For setup a device with IP address ``172.17.0.3`` and user ``admin`` via SSH:: POST /restconf/data/clixon-controller:devices HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:device": { "name":"test", "enabled":"true", "conn-type":"NETCONF_SSH", "user":"admin", "addr":"172.17.0.3" } } You can also configure device-groups and device-profiles as described in :ref:`the CLI tutotial `, for example. Transactions ============ Many of the controller's RPCs return a `transaction-id` that indicates that the result of the RPC is not immediately available. Instead, it indicates that a new transaction has been created. .. note:: RPCs returning a transaction-id are asynchronous Transactions can be monitored in one of the following ways: - Register and wait for a notification, as described in Section `notifications`_. - Sleep/Poll and read the status of the resulting action, such as the connection status, see Section `verify connection`_. - Sleep/Poll and read the status of the transaction using GET. To get the status of transaction "6" using GET send the following request:: POST /restconf/data/clixon-controller:transactions/transaction=6 HTTP/1.1 Accept: application/yang-data+json A typical reply:: HTTP/1.1 200 { "clixon-controller:transaction": [ { "tid": "6", "result": "SUCCESS", <--- ... } ] } Controller RPCs that create transactions are: - ``config-pull`` - ``controller-commit`` - ``connection-change`` - ``device-template-apply`` (of type RPC) Transactions are described in more detail in the :ref:`Transaction section`. Connect ======= If you have setup the configuration for your devices and installed the SSH keys, you can start connecting to them. For this, you need to invoke the ``connection-change`` RPC which starts a `device connect` transaction. .. note:: You need to install SSH keys before connection establishment The connection-change RPC takes a device or device-group as input and an operation. Example:: POST /restconf/operations/clixon-controller:connection-change HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device":"*", "operation":"OPEN" } } With the following reply containing a transaction-id:: HTTP/1.1 200 { "clixon-controller:output":{ "tid":"4" } } Verify connection ----------------- One way to verify a connection (apart from monitoring the transaction itself) is to wait and check the status of the connection using GET, as follows:: GET /restconf/data/clixon-controller:devices/device=openconfig1/conn-state HTTP/1.1 Accept: application/yang-data+json HTTP/1.1 200 { "clixon-controller:conn-state":"OPEN" } Select devices -------------- You can select devices in the connect RPCs as follows: - All devices: ``device: *`` - Individual device: ``device: openconfig1`` - Device pattern: ``device: openconfig*`` - Device-groups: ``device-group: mygroup`` - Device-group pattern: ``device-group: my*`` Connect operations ------------------ The operation in the initial example is `OPEN`. The operations are: - Establish connections to a set of devices: ``OPEN`` - Close connections: ``CLOSE`` - Close and the re-open connections: ``RECONNECT`` Example, reconnect to all devices in device-groups starting with "my*":: POST /restconf/operations/clixon-controller:connection-change HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device-group": "my*", "operation": "RECONNECT" } } Accessing device config ======================= When devices are open, you can get, put and push device configuration. GET device config ----------------- You can GET configuration from a single device as follows:: GET /restconf/data/clixon-controller:devices/device=openconfig1/config HTTP/1.1 Accept: application/yang-data+json HTTP/1.1 200 { "clixon-controller:config": { "openconfig-interfaces:interfaces": { "interface": [ { "name": "x", ... You can also get more specific config:: GET /restconf/data/clixon-controller:devices/device=openconfig1/\ config//openconfig-interfaces:interfaces/interface=x/config/type HTTP/1.1 Accept: application/yang-data+json HTTP/1.1 200 { "openconfig-interfaces:type": "iana-if-type:ethernetCsmacd" } PUT device config ----------------- To edit device configuration, use PUT, POST or PATCH and then `push` the changes to devices. With RESTCONF, modifications are written to the running datastore in the controller (local commit). Thereafter, the changes are pushed to the devices using the ``controller-commit`` RPC. For example, change the description of an interface using PUT:: PUT /restconf/data/clixon-controller:devices/device=openconfig1/config/\ openconfig-interfaces:interfaces/interface=x/config HTTP/1.1 Content-Type: application/yang-data+json { "openconfig-interfaces:config": { "name": "x", "type": "iana-if-type:ethernetCsmacd", "description": "My description" } } HTTP/1.1 204 PUSH device config ------------------ Thereafter, push the changes to a device using the ``controller-commit`` RPC:: POST /restconf/operations/clixon-controller:controller-commit HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device": "openconfig1", "push": "COMMIT", } } This may generate a reply as follows:: HTTP/1.1 200 { "clixon-controller:output":{ "tid":"3" } } Again, this starts an asynchronous transaction which can be monitored with methods described in Section `transactions`_. PULL device config ------------------ To synchronize, that is to pull the device config to the controller, use config-pull:: POST /restconf/operations/clixon-controller:config-pull HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device": "openconfig1", } } Notifications ============= The controller uses notifications to get asynchronous notifications and event streams. .. note:: Notifications are not fully functional in FCGI mode For example, connection establishment as described in Section `connect`_ and commit described in Section `push device config`_ create transactions. If you want to wait for such a transaction to complete, you can register for that event stream as follows:: GET /streams/controller-transaction HTTP/1.1 Accept: text/event-stream Cache-Control: no-cache Connection: keep-alive The `data` notification is an "SSE" / long poll event, which means that the call blocks and waits for notifications to be received:: HTTP/2 201 content-type: text/event-stream data: 2025-03-06T15:30:16.710209Z 4 clicon SUCCESS This means that a programmer needs to create a separate session apart from the original RPC: One which waits for a notification, and one which creates the transaction using an operation. Services ======== You may wish to read the :ref:`the service tutotial ` before reading this section. You can initate a service typically implemented in Python(PyAPI) by editing a service configuration and then applying the service using the ``controller-commit`` RPC. With CLI or NETCONF it is possible to edit a service in the candidate datastore and then only trigger the services that have changed. This is not possible in RESTCONF, since it does not use the candidate datastore. Instead you need to explicitly set which service has changed (or all) Edit a service config --------------------- First, edit a service. You can skip this part of you just want to trigger a service unconditionally. In the following example, edit the ``bar`` instance of the ``testA`` service in module ``myyang`` (ie instance ``testA[a_name='bar']``. :: POST /restconf/restconf/data/clixon-controller:services HTTP/1.1 Content-Type: application/yang-data+json { "myyang:testA": [ { "a_name": "bar", "params": [ "AA" ] } ] } This changes the service config in the running datastore of the controller. But it does not trigger a service. Getting service diff -------------------- To get the configuration that are going to be implemented on the devices, you need to send two RPCs to the controller. The first one is a ``controller-commit`` and the second one is a ``datastore-diff`` The operation is:: POST /restconf/operations/clixon-controller:controller-commit HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "push": "NONE", "actions": "FORCE", "source": "ds:candidate" "service-instance":"testA[a_name='bar']" } } and then the diff can be seen by running:: POST /restconf/operations/clixon-controller:datastore-diff HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device": "*", "config-type1": "RUNNING", "config-type2": "ACTIONS" } } This should be done before the service code is triggered. Trigger service code -------------------- To trigger a service, you need to send a ``controller-commit`` RPC to the controller. You can do this either after editing a service, or unconditionally, such as in a periodic process. In the following example, you trigger the service as follows: - It applies for all devices: ``device:*``. - You push and commit the service result to the devices: ``push:COMMIT``. You could also just ``VALIDATE`` the service in the devices - The service is unconditionally run: ``actions:FORCE``. This is the only option for RESTCONF. - The datastore to work with is ``source:candidate``. This is also necessary for RESTCONF. - Trigger the service for a specific instance: ``service-instance:testA[a_name='bar']``. You may also skip this field to trigger all services. The corresponding operation is:: POST /restconf/operations/clixon-controller:controller-commit HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "push": "COMMIT", "actions": "FORCE", "source": "ds:candidate" "service-instance":"testA[a_name='bar']" } } Where a transaction id is returned:: HTTP/1.1 200 { "clixon-controller:output":{ "tid":23 } } Delete service -------------- To remove a service using RESTCONF you have to do it in two steps. First, similar to in Section `Trigger service code`_ but instead of sending `actions:FORCE`, you send `actions:DELETE` to remove device configuration created by the service. Then remove the service definition in a second step. The operation to remove the device configuration created by a service:: POST /restconf/operations/clixon-controller:controller-commit HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "push": "COMMIT", "actions": "DELETE", "source": "ds:candidate" "service-instance":"testA[a_name='bar']" } } And then remove the service definition itself:: DELETE /restconf/data/myyang:testA=a_name='bar' HTTP/1.1 Device RPCs =========== You can send an RPC to devices via the controller using the ``device-rpc`` RPC, and get the result with ``device-rpc-result``. Send RPC -------- Example of sending the RPC using JSON:: POST /restconf/operations/clixon-controller:device-rpc HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device": "openconfig*", "config": { "clixon-lib:stats": { "modules": "true" } } } } Where a transaction id is returned:: HTTP/1.1 200 { "clixon-controller:output":{ "tid":"5" } } Check transaction state ----------------------- A transaction has been created and the client needs to wait for results via a notification (see Section `notifications`_) or poll for completion of the transaction:: GET /restconf/data/clixon-controller:transactions/transaction=5 HTTP/1.1 Accept: application/yang-data+json The reply could be:: HTTP/1.1 200 { "clixon-controller:transaction": [ { "tid": "6", "username": "clicon", "result": "SUCCESS", ... Read result ----------- If the transaction has completed successfuly, you can read the results via the ``rpc-device-result`` RPC:: POST /restconf/operations/clixon-controller:device-rpc-result HTTP/1.1 Content-Type: application/yang-data+json Accept: application/yang-data+json { "clixon-controller:input": { "tid":"11", } } Example reply:: HTTP/1.1 200 { "clixon-controller:output": { "tid": "11", "devices": { "devdata": [ { "name": "openconfig1", "data": { "global": { "xmlnr": "1570", "yangnr": "166357" ... Note the ``devdata`` field which returns the reply from the RPC. That is, the reply for the ``stats`` RPC to ``openconfig1`` is:: "data": { "global": { "xmlnr": "1570", "yangnr": "166357" The ``devdata`` field may contain replies from multiple devices. Send using XML -------------- Instead of using JSON in the rpc-template body, you can also use XML:: POST /restconf/operations/clixon-controller:device-rpc HTTP/1.1 Content-Type: application/yang-data+xml openconfig* Get device state ================ You can get state data from device by using a device RPC as a workaround for not supplying it with a top-level `GET`. Device state using XML ---------------------- Example, get the ssh state of all openconfig devices:: POST /restconf/operations/clixon-controller:device-rpc HTTP/1.1 Content-Type: application/yang-data+xml openconfig* HTTP/1.1 200 { "clixon-controller:output":{ "tid":"6" } } where the `filter` statement selects the ``system/ssh-server`` state. Polling for successful result:: GET /restconf/data/clixon-controller:transactions/transaction=6 HTTP/1.1 Accept: application/yang-data+json A reply could be:: HTTP/2 200 content-type: application/yang-data+json { "clixon-controller:transaction": [ { "tid": "8", "username": "anonymous", "result": "SUCCESS", ... Get the result:: POST /restconf/operations/clixon-controller:device-rpc-result HTTP/1.1 Content-Type: application/yang-data+json Accept: application/yang-data+json { "clixon-controller:input": { "tid":"8", } } The state result reply:: HTTP/1.1 200 { "clixon-controller:output": { "tid": "8", "devices": { "devdata": [ { "name": "openconfig1", "data": { "data": { "system": { "ssh-server": { "state": { "enable": "true", "protocol-version": "V2" } } } } } }, ... where the result of the first matchong device (``openconfig1``) is shown. Device state using JSON ----------------------- At this time it is not possible to get a subset of subset of state data for JSON, ie an XPath selection, the whole state data is returned. .. note:: You cannot get individual elements via JSON, just ALL device state Example, get all state of all "openconfig*" devices:: POST /restconf/operations/clixon-controller:device-rpc HTTP/1.1 Content-Type: application/yang-data+json { "clixon-controller:input": { "device":"openconfig*", "config": { "ietf-netconf:get":null } } } HTTP/1.1 200 { "clixon-controller:output":{ "tid":"6" } } Polling for result:: GET /restconf/data/clixon-controller:transactions/transaction=6 HTTP/1.1 Accept: application/yang-data+json where the result would be the complete state of all matching devices.