ConfD User Guide
   Next

ConfD User Guide

Tail-f Systems

ConfD 6.4

Copyright © 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 Tail-f Systems

All contents in this document are confidential and proprietary to Tail-f Systems.

March 21, 2017

Table of Contents

1. About the Documentation
1.1. How to Read This Guide
1.2. Getting Documentation
1.3. Formatting Conventions
1.4. Documentation Feedback
2. An introduction to ConfD
2.1. An on-device software system for configuration management
2.2. ConfD Architecture
3. The YANG Data Modeling Language
3.1. The YANG Data Modeling Language
3.2. YANG in ConfD
3.3. YANG Introduction
3.4. Working With YANG Modules
3.5. Integrity Constraints
3.6. The when statement
3.7. Using the Tail-f Extensions with YANG
3.8. Custom Help Texts and Error Messages
3.9. Hidden Data
3.10. An Example: Modeling a List of Interfaces
3.11. More on leafrefs
3.12. Using Multiple Namespaces
3.13. Module Names, Namespaces and Revisions
3.14. Hash Values and the id-value Statement
3.15. Migrating from Confspecs to YANG
3.16. The pyang tool
4. Rendering Agents
4.1. Introduction
4.2. Data Model
4.3. Using the CLIs
4.4. Using NETCONF
5. CDB - The ConfD XML Database
5.1. Introduction
5.2. CDB
5.3. An example
5.4. Using keypaths
5.5. A session
5.6. CDB subscriptions
5.7. Reconnect
5.8. Loading initial data into CDB
5.9. Automatic schema upgrades and downgrades
5.10. Using initialization files for upgrade
5.11. Using MAAPI to modify CDB during upgrade
5.12. More complex schema upgrades
5.13. The full dhcpd example
6. Operational Data
6.1. Introduction to Operational Data
6.2. Reading Statistics Data
6.3. Callpoints and Callbacks
6.4. Data Callbacks
6.5. User Sessions and ConfD Transactions
6.6. C Example with Operational Data
6.7. The Protocol and a Library Threads Discussion
6.8. Operational data in CDB
6.9. Delayed Replies
6.10. Caching Operational Data
6.11. Operational data lists without keys
7. The external database API
7.1. Introduction to external data
7.2. Scenario - The database is a file
7.3. Callpoints and callbacks
7.4. Data Callbacks
7.5. User sessions and ConfD Transactions
7.6. External configuration data
7.7. External configuration data with transactions
7.8. Writable operational data
7.9. Supporting candidate commit
7.10. Discussion - CDB versus external DB
8. Configuration Meta-Data
8.1. Introduction to Configuration Meta-Data
8.2. Meta-Data: annotation
8.3. Meta-Data: tag
8.4. Meta-Data: inactive
9. Semantic validation
9.1. Why Do We Need to Validate
9.2. Syntactic Validation in YANG models
9.3. Integrity Constraints in YANG Models
9.4. The YANG must Statement
9.5. Validation Logic
9.6. Validation Points
9.7. Validating Data in C
9.8. Validation Points and CDB
9.9. Dependencies - Why Does Validation Points Get Called
9.10. Configuration Policies
10. Transformations, Hooks, Hidden Data and Symlinks
10.1. Introduction
10.2. Transformation Control Flow
10.3. An Example
10.4. AAA Transform
10.5. Other Use Cases for Transformations
10.6. Hooks
10.7. Hidden Data
10.8. tailf:symlink
11. Actions
11.1. Introduction
11.2. Action as a Callback
11.3. Action as an Executable
11.4. Related functionality
12. Notifications
12.1. ConfD Asynchronous Events
12.2. Audit Messages
12.3. Syslog Messages
12.4. Commit Events
12.5. Commit Failure Events
12.6. Confirmed Commit Events
12.7. Commit Progress Events
12.8. User Sessions
12.9. High Availability - Cluster Events
12.10. Subagent Events
12.11. SNMP Agent Audit Log
12.12. Forwarding Events
12.13. In-service Upgrade Events
12.14. Heartbeat and Health Check Events
12.15. Notification stream Events
13. In-service Data Model Upgrade
13.1. Introduction
13.2. Preparing for the Upgrade
13.3. Initializing the Upgrade
13.4. Performing the Upgrade
13.5. Committing the Upgrade
13.6. Aborting the Upgrade
13.7. Upgrade and HA
14. The AAA infrastructure
14.1. The problem
14.2. Structure - data models
14.3. AAA related items in confd.conf
14.4. Authentication
14.5. Group Membership
14.6. Authorization
14.7. The AAA cache
14.8. Populating AAA using CDB
14.9. Populating AAA using external data
14.10. Hiding the AAA tree
15. The NETCONF Server
15.1. Introduction
15.2. Capabilities
15.3. NETCONF Transport Protocols
15.4. Configuration of the NETCONF Server
15.5. Extending the NETCONF Server
15.6. Monitoring of the NETCONF Server
15.7. Notification Capability
15.8. Using netconf-console
15.9. Actions Capability
15.10. Transactions Capability
15.11. Proxy Forwarding Capability
15.12. Inactive Capability
15.13. Tail-f Identification Capability
15.14. The Query API
15.15. Meta-data in Attributes
15.16. Namespace for Additional Error Information
16. The CLI agent
16.1. Overview
16.2. The J-style CLI
16.3. The C- and I-style CLI
16.4. The CLI in action
16.5. Environment for OS command execution
16.6. Command output processing
16.7. Range expressions
16.8. Autorendering of enabled/disabled
16.9. Actions
16.10. Command history
16.11. Command line editing
16.12. Using CLI completion
16.13. Using the comment characters # or !
16.14. Annotations and tags
16.15. Activate and Deactivate
16.16. CLI messages
16.17. confd.conf settings
16.18. CLI Environment
16.19. Commands in J-style
16.20. Commands in C/I-style
16.21. Customizing the CLI
16.22. User defined wizards
16.23. User defined wizards in C
16.24. User defined commands in C using the C-API
16.25. User defined commands as shell scripts
16.26. Modifying built-in commands
16.27. Tailoring show commands
16.28. Change password at initial login
17. The SNMP Agent
17.1. Introduction to the ConfD SNMP Agent
17.2. Agent Functional Description
17.3. Generating MIBs from YANG
17.4. Configuring the SNMP Agent
17.5. How the SNMP Agent Interacts with ConfD
17.6. Running the SNMP Agent as a NET-SNMP subagent
18. Web UI Development
18.1. Introduction
18.2. Example of a common flow
18.3. Example of a JSON-RPC client
18.4. Example of a Comet client
19. The JSON-RPC API
19.1. JSON-RPC
19.2. Methods - commands
19.3. Methods - commands - subscribe
19.4. Methods - data
19.5. Methods - data - attrs
19.6. Methods - data - leafs
19.7. Methods - data - leafref
19.8. Methods - data - lists
19.9. Methods - data - query
19.10. Methods - database
19.11. Methods - general
19.12. Methods - messages
19.13. Methods - rollbacks
19.14. Methods - schema
19.15. Methods - session
19.16. Methods - session data
19.17. Methods - transaction
19.18. Methods - transaction - changes
19.19. Methods - transaction - commit changes
19.20. Methods - transaction - webui
20. The web server
20.1. Introduction
20.2. Web server capabilities
20.3. CGI support
20.4. Proxy server example
21. The REST API
21.1. Introduction
21.2. Getting started
21.3. Resource Examples
21.4. Resources
21.5. Configuration Meta-Data
21.6. Request/Response headers
21.7. Special characters
21.8. Error Responses
21.9. The Query API
21.10. Custom Response HTTP Headers
21.11. HTTP Status Codes
22. The RESTCONF API
22.1. Introduction
22.2. Getting started
22.3. Schema resource
22.4. Extensions
22.5. Deviations
23. The Management Agent API
23.1. What is MAAPI?
23.2. A custom toy CLI
24. High Availability
24.1. Introduction to ConfD High Availability
24.2. HA framework requirements
24.3. Mode of operation
24.4. Security aspects
24.5. API
24.6. Ticks
24.7. Joining a cluster
24.8. Relay slaves
24.9. CDB replication
25. The SNMP Gateway
25.1. Introduction to the ConfD SNMP Gateway
25.2. Configuring Agent Access
25.3. Compiling the MIBs
25.4. Receiving and Forwarding Notifications
25.5. Example Scenario
26. Subagents and Proxies
26.1. Introduction
26.2. Subagent Registration
26.3. Subagent Requirements
26.4. Proxies
27. Plug-and-play scripting
27.1. Introduction
27.2. Script storage
27.3. Script interface
27.4. Loading of scripts
27.5. Command scripts
27.6. Policy scripts
27.7. Post-commit scripts
28. Advanced Topics
28.1. Datastores
28.2. Locks
28.3. Installing ConfD on a target system
28.4. Configuring ConfD
28.5. Starting ConfD
28.6. ConfD IPC
28.7. Restart strategies
28.8. Security issues
28.9. Running ConfD as a non privileged user
28.10. Storing encrypted values in ConfD
28.11. Disaster management
28.12. Troubleshooting
28.13. Tuning the size of confd_hkeypath_t
28.14. Error Message Customization
28.15. Using a different version of OpenSSL
28.16. Using shared memory for schema information
28.17. Running application code inside ConfD
I. ConfD man-pages, Volume 1
confd — command to start and control the ConfD daemon
confd_aaa_bridge — Populating ConfD aaa_bridge.fxs with external data
confd_cli — Frontend to the ConfD CLI engine
confd_cmd — Command line utility that interfaces to common ConfD library functions
confd_load — Command line utility to load and save ConfD configurations
confdc — Confdc compiler
maapi — command to access an ongoing transaction
pyang — validate and convert YANG modules to various formats
II. ConfD man-pages, Volume 3
confd_lib — C library for connecting to ConfD
confd_lib_cdb — library for connecting to ConfD built-in XML database (CDB)
confd_lib_dp — callback library for connecting data providers to ConfD
confd_lib_events — library for subscribing to ConfD event notifications
confd_lib_ha — library for connecting to ConfD HA subsystem
confd_lib_lib — common library functions for applications connecting to ConfD
confd_lib_maapi — MAAPI (Management Agent API). A library for connecting to ConfD with a read/write interface inside transactions.
confd_types — ConfD value representation in C
III. ConfD man-pages, Volume 5
clispec — CLI specification file format
confd.conf — ConfD daemon configuration file format
mib_annotations — MIB annotations file format
tailf_yang_cli_extensions — Tail-f YANG CLI extensions
tailf_yang_extensions — Tail-f YANG extensions
Glossary

List of Tables

3.1. YANG built-in types
17.1. SMI mapping to YANG types
17.2. YANG mapping to SMI types
21.1. REST vs NETCONF operations
21.2. Query Parameters
21.3. Resources and their Media Types
21.4. Fields of the /api resource
21.5. Fields of the /api/<datastore> resource
21.6. Built in operations
21.7. Error code vs HTTP Status
28.1. ConfD Start Phases
28.2. ConfD Start Phases, running in foreground

List of Examples

5.1. a simple server data model, servers.yang
5.2. Pseudo code showing several sessions reusing one connection
5.3. Pseudo code demonstrating how to avoid re-reading the configuration
5.4. Version 1.0 of the forest module
5.5. Initial forest instance document
5.6. Version 2.0 of the forest module
5.7. Forest instance document after upgrade
5.8. Enabling the developer log
5.9. Developer log entries resulting from upgrade
5.10. Version 1.5 of the servers.yang module
5.11. Writing to an upgrade transaction using MAAPI
5.12. Version 2 of the servers.yang module
5.13. The upgrade() function of server_upgrade.c
5.14. A YANG module describing a dhcpd server configuration
6.1. netstat.yang
6.2. ARP table YANG module
6.3. Populated ARP table
7.1. A list of server structures
7.2. The smp.yang module
7.3. get_next() callback for smp.yang
7.4. create() callback for smp.yang
7.5. remove() callback for smp.yang
7.6. set_elem() callback for smp.yang
7.7. save() utility function
7.8. write callbacks using accumulate
7.9. prepare() callback using the accumulated write ops
7.10. commit() and abort()
7.11. Code to restore our array from a file
7.12. checkpoint db callbacks
10.1. full.yang
10.2. small.yang
10.3. users.yang
12.1. Creating a notification socket
12.2. reading the audit data
15.1. Example math rpc
17.1. Simple YANG module
17.2. Generating and compiling YANG from MIB
17.3. The YANG file generated by confdc --mib2yang
17.4. Specifying built-in MIBs to be loaded into the agent
17.5. SMI definition of an optional object
17.6. YANG definition of an optional leaf
17.7. simple.mib
17.8. simple.yang
17.9. simple.yang with secondary index
17.10. TruthValue from the SNMPv2-TC
17.11. A typedef for TruthValue
17.12. Functions for sending notification from C
17.13. SNMP varbind structures from confd_maapi.h
17.14. Notification registration
17.15. Sending a coldStart notification
17.16. Sending a notification with a varbind
17.17. Example of a confd.conf
17.18. Old confd.conf content
17.19. Updated confd.conf content
17.20. Example community_init.xml
19.1. Method get_value
19.2. Method set_value
19.3. Method query
19.4. Method start_query
19.5. Method run_query
19.6. Method reset_query
19.7. Method stop_query
19.8. Method comet
19.9. Method get_schema
19.10. Method run_action
19.11. Method login
19.12. Method logout
19.13. Method get_trans
19.14. Method new_trans
19.15. Method get_trans_changes
21.1. ConfD configuration for REST
21.2. Request URI structure
21.3. Using curl for accessing ConfD
21.4. Get the "sys/interfaces" resource represented as JSON
21.5. Create a new "sys/routes/inet/route" resource, with JSON payload
21.6. Replace the "sys/routes/inet/route" resource contents
21.7. Update the "sys/routes/inet/route" resource contents
21.8. Delete the "sys/routes/inet/route" resource contents
21.9. Get options for the "sys" resource
21.10. Get head for the "sys/interfaces/ex:serial" resource
21.11. Shallow get for the "sys" resource
21.12. Deep get for the "sys/interfaces/interface" resource
21.13. Limit the response
21.14. Limit the response with select
21.15. The "sys/ntp/server" list (no defaults)
21.16. The "sys/ntp/server" list with all defaults
21.17. Creating a "sys/routes/inet/route" resource
21.18. Creating a "sys/interfaces/serial/ppp0/multilink" resource
21.19. Creating a "route" resource using PUT
21.20. The "route" resource after creation
21.21. Replacing a "route" resource using PUT
21.22. The "route" resource after replace
21.23. Creating a "sys/interfaces/serial/ppp0/multilink" resource
21.24. Creating a "sys/interfaces/serial/ppp0/authentication" resource
21.25. The "authentication" resource after replace
21.26. Updating a "route" resource using PATCH
21.27. The "route" resource after update
21.28. Creating a "sys/interfaces/serial/ppp0/multilink" resource
21.29. Creating a "sys/interfaces/serial/ppp0/authentication" resource
21.30. The "authentication" resource after update
21.31. The "sys/dns/server" list before insert
21.32. Insert=before in the "sys/dns/server" list
21.33. The "sys/dns/server" list after insert
21.34. An "archive-log" action request example
21.35. delete the "sys/interfaces/ex:serial" list
21.36. The "sys/interfaces" resource after delete
21.37. delete the "sys/interfaces/ex:serial" list with rollback label and comment
21.38. Namespaces in JSON
21.39. GET the /api resource
21.40. GET the /api/running resource
21.41. Action in /api/operational
21.42. GET rollback files information
21.43. GET rollback file content
21.44. Find and use the rollback operation resource
21.45. XML representation of meta-data
21.46. JSON representation of meta-data
21.47. Example of a XML formatted error message
21.48. Example of a JSON formatted error message
21.49. ConfD configuration for REST
21.50.
22.1. ConfD configuration for REST
22.2. ConfD configuration for RESTCONF
22.3. ConfD RESTCONF capabilities
22.4. Use of collections
22.5. Use of collections
23.1. scli.yang YANG module
24.1. A data model divided into common and node specific subtrees
25.1. Example snmpgw configuration fragment in confd.conf
25.2. C code for registering reception of notifications
25.3. Example 1 of translating and compiling a MIB
26.1. smtp subagent data
26.2. imap and pop subagent data
26.3. Equipment subagent data
26.4. master agent data
26.5. Compile the YANG modules at the master
26.6. Master agent's confd.conf
26.7. Proxy configuration
26.8. Agent replies with forward capability
26.9. Manager issues forward rpc to board-1
26.10. Manager issues command
26.11. close-session
26.12. Auto login
26.13. Forward rpc with auth data
144. Reloading all xml files in the cdb directory
145. Merging in the contents of conf.cli
146. Print interface config and statistics data in cli format
147. Using xslt to format output
148. Using xmllint to pretty print the xml output
149. Saving config and operational data to /tmp/conf.xml
150. Restoring both config and operational data
151. Measure how long it takes to fetch config
152. Output all instances in list /foo/table which has ix larger than 10
153. confd-light.cli
154. The servers YANG model
   Next
   Chapter 1. About the Documentation