Chapter 4: Python Network Automation with ncclient
Learning Objectives
Build Python scripts using ncclient to manage Cisco IOS XE devices via NETCONF
Construct and send NETCONF RPC operations including get, get-config, and edit-config
Use XML filters — subtree and XPath — to retrieve targeted configuration and state data
Implement configuration validation and commit workflows, including confirmed commits and rollback
Section 1: ncclient Fundamentals
Pre-Quiz — Section 1: ncclient Fundamentals
1. What TCP port does NETCONF use by default?
2. Which ncclient function is used to establish a NETCONF session to a device?
3. What IOS XE configuration command enables the NETCONF/YANG subsystem?
4. Why is using manager.connect() as a context manager (with statement) preferred?
5. Where are NETCONF server capabilities exchanged during session setup?
NETCONF (Network Configuration Protocol, RFC 6241) is an XML-based RPC protocol that communicates over SSH on TCP port 830. It gives automation scripts a vendor-neutral, schema-validated interface to device configuration and operational state. ncclient is the de facto standard Python library for NETCONF client development on the Cisco ENAUTO 300-435 exam.
During session setup both sides exchange <hello> messages advertising capability URNs. Always check capabilities before using advanced features:
with manager.connect(**device) as m:
caps = list(m.server_capabilities)
has_candidate = any("candidate:1.0" in c for c in caps)
has_validate = any("validate:1.1" in c for c in caps)
has_xpath = any("xpath:1.0" in c for c in caps)
has_conf_cmmt = any("confirmed-commit:1.1" in c for c in caps)
NETCONF Session Lifecycle
sequenceDiagram
participant Script as Python Script (ncclient)
participant Device as IOS XE Device (port 830)
Script->>Device: TCP SYN → SSH Handshake
Device-->>Script: SSH Session Established
Script->>Device: NETCONF <hello> (client capabilities)
Device-->>Script: NETCONF <hello> (server capabilities list)
Note over Script,Device: Session negotiated — Manager object ready
loop NETCONF Operations
Script->>Device: <rpc> get / get-config / edit-config / etc.
Device-->>Script: <rpc-reply> with <data> or <ok/> or <rpc-error>
end
alt Normal teardown (context manager __exit__)
Script->>Device: <close-session/>
Device-->>Script: <ok/>
else Abnormal termination
Note over Device: SSH keepalive timeout — auto-release locks
end
NETCONF Session Handshake (animated)
Python Script (ncclient)
TCP SYN + SSH
SSH Established
<hello> client caps
<hello> server caps
<rpc> operations
<rpc-reply> / <ok/>
IOS XE Device (port 830)
Key Points — Section 1
NETCONF operates over SSH on TCP port 830; it is XML-based and defined in RFC 6241.
Enable NETCONF on IOS XE with netconf-yang and verify that ncsshd is running.
Use manager.connect() as a context manager so close_session() is guaranteed even on exceptions.
The device_params={"name": "iosxe"} argument hints at vendor-specific protocol quirks; always include it for IOS XE.
2. Which ncclient function is used to establish a NETCONF session to a device?
3. What IOS XE configuration command enables the NETCONF/YANG subsystem?
4. Why is using manager.connect() as a context manager (with statement) preferred?
5. Where are NETCONF server capabilities exchanged during session setup?
Section 2: NETCONF Operations with ncclient
Pre-Quiz — Section 2: NETCONF Operations
1. What is the key difference between get and get_config?
2. What is the XML root element that must wrap an edit_config payload?
3. What happens if you call lock() and the datastore is already locked by another session?
4. What is the default_operation="replace" behavior for edit_config?
5. On IOS XE, after a successful commit(), what additional step is needed to persist changes across reload?
get_config vs. get
get_config(source, filter=None) retrieves configuration from a named datastore ("running", "candidate", or "startup"). get(filter=None) returns both configuration and live operational state — interface counters, routing table entries, runtime statistics.
edit_config(target, config) sends an <edit-config> RPC. The config payload must be wrapped in a <config> root element (not <data>). The default_operation parameter controls merge behavior:
default_operation
Behavior
"merge" (default)
New values replace old; unmentioned nodes are retained
"replace"
Replace the entire target subtree with the provided config
"none"
Only nodes with an explicit operation attribute are changed
Per-element operation attributes: merge, replace, create, delete, remove. Note that delete raises an error if the node does not exist; remove silently succeeds.
Lock / Unlock Pattern
with manager.connect(**device) as m:
m.lock("candidate")
try:
m.edit_config(target="candidate", config=config_payload)
m.commit()
finally:
m.unlock("candidate") # always release
Committing and Saving to Startup
After commit(), IOS XE updates running config but not startup. To persist across reload, dispatch the vendor-specific save-config RPC:
from ncclient.xml_ import to_ele
save_rpc = to_ele('<save-config xmlns="http://cisco.com/yang/cisco-ia"/>')
m.dispatch(save_rpc)
Key Points — Section 2
get retrieves config + operational state; get_config retrieves configuration only from a named datastore.
Always wrap edit_config payloads in a <config> root element, not <data>.
Always wrap lock() / unlock() in a try/finally block — a lock left held blocks all other sessions including CLI.
On IOS XE, commit() updates running but not startup; dispatch save-config to persist across reload.
An RPCError with error-tag: in-use means the datastore is locked by another session; the error-info field contains the lock holder's session ID.
Post-Quiz — Section 2: NETCONF Operations
1. What is the key difference between get and get_config?
2. What is the XML root element that must wrap an edit_config payload?
3. What happens if you call lock() and the datastore is already locked by another session?
4. What is the default_operation="replace" behavior for edit_config?
5. On IOS XE, after a successful commit(), what additional step is needed to persist changes across reload?
Section 3: XML Filtering and Data Retrieval
Pre-Quiz — Section 3: XML Filtering
1. How are filters passed to get_config() in ncclient?
2. In a subtree filter, what role does a content match node serve?
3. What capability URN must a device advertise before XPath filtering can be used?
Requesting the full configuration from a production device can return 50,000+ lines of XML. Filters let the server extract only what you need before sending the reply, saving bandwidth, memory, and device CPU.
ncclient accepts filters as a two-element tuple: ("subtree", xml_string) or ("xpath", expression).
Subtree Filter Component Types
Type
XML Form
Behavior
Namespace selection
xmlns="..." attribute
Constrains match to a specific YANG module
Containment node
Element with children, no text
Navigates deeper into the YANG tree
Selection node
Empty self-closing tag
Return this node and all descendants
Content match node
Element with text value
Equality predicate — filters list entries
Combined
Content match + selection siblings
Filter to a specific entry, select specific leaves
# Content match: return ONLY GigabitEthernet1 and its enabled/ipv4 leaves
interface_filter = """
<interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
<interface>
<name>GigabitEthernet1</name> <!-- content match -->
<enabled/> <!-- selection node -->
<ipv4 xmlns="urn:ietf:params:xml:ns:yang:ietf-ip"/>
</interface>
</interfaces>
"""
reply = m.get_config(source="running", filter=("subtree", interface_filter))
flowchart TD
A([XML filter element\nencountered]) --> B{Has xmlns\nattribute?}
B -->|Yes| C[Namespace Selection\nConstrains to specific\nYANG module]
B -->|No| D{Has child\nelements?}
C --> D
D -->|Yes, no text| E[Containment Node\nNavigates deeper\ninto YANG tree]
D -->|Self-closing tag| F[Selection Node\nReturn this node\nand all descendants]
D -->|Yes, with text| G[Content Match Node\nEquality predicate:\nfilter list entries]
E --> H{Children mix\ntext and self-closing?}
H -->|Yes| I[Combined Filter\nMatch entry + select leaves]
H -->|No| D
style C fill:#1d3557,color:#fff
style E fill:#457b9d,color:#fff
style F fill:#2a9d8f,color:#fff
style G fill:#e76f51,color:#fff
style I fill:#6d6875,color:#fff
XPath Filtering
XPath offers richer predicate logic but requires the xpath:1.0 capability. Always verify before use:
ns_map = {
"ios": "http://cisco.com/ns/yang/Cisco-IOS-XE-native",
"if": "urn:ietf:params:xml:ns:yang:ietf-interfaces",
}
xpath_expr = "/if:interfaces/if:interface[if:name='GigabitEthernet1']/if:enabled"
with manager.connect(**device) as m:
if not any("xpath:1.0" in c for c in m.server_capabilities):
raise RuntimeError("Device does not support XPath filtering")
reply = m.get_config(source="running", filter=("xpath", (ns_map, xpath_expr)))
Parsing Replies with lxml
ns = {"ios": "http://cisco.com/ns/yang/Cisco-IOS-XE-native"}
hostname = reply.data.find(".//ios:hostname", namespaces=ns).text
# Collect all interface names via xpath()
names = reply.data_ele.xpath("//if:interface/if:name/text()", namespaces=ns)
Key Points — Section 3
Unfiltered get_config is a performance anti-pattern; always apply a filter in production code.
Subtree filtering is mandated by RFC 6241 and universally supported; XPath filtering is optional and requires the xpath:1.0 capability.
A content match node (<name>GigabitEthernet1</name>) filters list entries; a selection node (<name/>) returns all entries.
Always include the correct xmlns in filter XML — without it, the server may not identify the correct YANG module.
Use reply.data_ele for lxml navigation and supply explicit namespace maps to .find() and .xpath() for correctness.
Post-Quiz — Section 3: XML Filtering
1. How are filters passed to get_config() in ncclient?
2. In a subtree filter, what role does a content match node serve?
3. What capability URN must a device advertise before XPath filtering can be used?
Section 4: Advanced ncclient Patterns
Pre-Quiz — Section 4: Advanced Patterns
1. What is a confirmed commit and what safety benefit does it provide?
2. What does discard_changes() do?
3. What NETCONF error tag indicates a YANG model constraint violation?
4. When candidate datastore is enabled on IOS XE, which capability is automatically disabled?
5. What function in ncclient.xml_ converts an XML string into an lxml Element?
The Candidate Datastore Workflow
The candidate datastore is a staging area for changes. When enabled on IOS XE, writable-running is automatically disabled — all writes must go through candidate → commit.
flowchart TD
A([Start]) --> B[lock candidate datastore]
B --> C[edit_config to candidate]
C --> D{validate candidate\nagainst YANG models}
D -->|Validation fails| E[discard_changes]
E --> F([Unlock and Abort])
D -->|Validation passes| G[commit confirmed=True\nstart rollback timer]
G --> H{Verify running config\nmatches intent}
H -->|Fails| I[Let timer expire]
I --> J([Auto-rollback restores\nprevious running config])
H -->|Passes| K[commit confirming\ncancels rollback timer]
K --> L[dispatch save-config]
L --> M[unlock candidate]
M --> N([Success])
style A fill:#2d6a4f,color:#fff
style N fill:#2d6a4f,color:#fff
style F fill:#9b2226,color:#fff
style J fill:#9b2226,color:#fff
Confirmed Commit
commit(confirmed=True, confirm_timeout=120) applies the change but starts a countdown. If your script cannot send the confirming commit() within 120 seconds — because the change broke connectivity — the device auto-rolls back.
sequenceDiagram
participant Script as Python Script
participant Device as IOS XE Device
Script->>Device: edit_config(target=candidate, ...)
Device-->>Script: <ok/>
Script->>Device: validate(source=candidate)
Device-->>Script: <ok/>
Script->>Device: commit(confirmed=True, confirm_timeout=120)
Device-->>Script: <ok/> — running updated, 120s timer starts
Note over Device: Rollback timer running
alt Confirming commit received in time
Script->>Device: commit()
Device-->>Script: <ok/> — timer cancelled, change permanent
else Timer expires (no confirming commit)
Note over Device: Auto-rollback to pre-commit running config
end
Production Candidate Workflow (animated)
lock(candidate)
→
edit_config
→
validate
→
commit(confirmed)
→
verify
→
commit()
→
save-config
On any RPCError: discard_changes() → unlock(candidate)
Structured Error Handling
from ncclient.operations import RPCError
try:
m.commit()
except RPCError as e:
print(f"Error tag : {e.tag}") # 'in-use', 'invalid-value', etc.
print(f"Error message : {e.message}")
print(f"Error info : {e.info}") # e.g. lock holder session ID
m.discard_changes()
finally:
m.unlock("candidate")
Error Tag
Cause
Resolution
in-use
Datastore locked by another session
Wait and retry; check e.info for lock holder session ID
invalid-value
YANG constraint violation
Fix the XML payload to comply with the YANG model
data-missing
delete on a non-existent node
Use remove for idempotent deletes
data-exists
create on an already-existing node
Use merge instead of create
operation-failed
Generic failure during commit
Inspect e.message for device-specific detail
Key Points — Section 4
The full production workflow is: lock → edit_config → validate → commit(confirmed=True) → verify → commit() → save-config → unlock.
Confirmed commits are essential for remote changes — if connectivity is lost during the confirmation window, the device auto-rolls back after the timeout.
validate() checks YANG model constraints before touching running config; if it fails, the candidate is preserved for correction.
discard_changes() resets candidate to match running — it does not touch the running config.
Always catch RPCError from ncclient.operations and inspect e.tag for structured diagnostics; use to_ele() and dispatch() for vendor-specific RPCs like save-config.
Post-Quiz — Section 4: Advanced Patterns
1. What is a confirmed commit and what safety benefit does it provide?
2. What does discard_changes() do?
3. What NETCONF error tag indicates a YANG model constraint violation?
4. When candidate datastore is enabled on IOS XE, which capability is automatically disabled?
5. What function in ncclient.xml_ converts an XML string into an lxml Element?