Deep Dive into NetworkManager – A Comprehensive Guide to NetworkManager in Linux

Introduction to NetworkManager in Linux

NetworkManager is a system service that manages network devices and connections on Red Hat Enterprise Linux (RHEL). It automatically creates connection profiles for each Ethernet adapter in a host and, by default, uses DHCP for both IPv4 and IPv6 connections.

NetworkManager supports various network types, including Ethernet, bonds, teams, VLANs, bridges, and VPNs.

Benefits of NetworkManager

NetworkManager offers several benefits:

• Simplified Network Management: NetworkManager aims to simplify the configuration and management of network settings for both administrators and users.
• Automatic Configuration: NetworkManager automatically detects and configures network devices when they are added to a system, eliminating the need for manual configuration in many cases.
• Consistent Network Interface Naming: NetworkManager employs the udev device manager for consistent network interface naming, ensuring that devices retain their names across system reboots and hardware changes.
• Multiple Configuration Methods: RHEL offers various methods to configure NetworkManager, including the command-line (nmcli), text-based UI (nmtui), graphical interfaces (GNOME Settings and nm-connection-editor), the Nmstate API (nmstatectl), and Ansible playbooks using RHEL system roles.
• Support for Advanced Networking Features: NetworkManager supports advanced networking features, such as bonding, VLAN tagging, bridging, VPN connections (IPsec and WireGuard), and policy-based routing.
• Integration with Other System Components: NetworkManager integrates with other system components, such as systemd and firewalld, to provide a unified and coordinated approach to network management.
• Offline Configuration: You can create NetworkManager connection profiles in offline mode, which can be beneficial for pre-deploying configurations or working in environments without the NetworkManager service running.

Key Concepts in NetworkManager

• Connection Profiles: NetworkManager manages network settings through connection profiles. These profiles store information such as the network interface name, IP address settings, DNS server information, and VPN configuration. You can create multiple profiles to connect to different networks.
• Network Devices: NetworkManager recognizes and manages various network device types, including physical and virtual Ethernet adapters, bonds, teams, VLANs, bridges, and loopback interfaces.
• Nmstate API: Nmstate is a declarative API that provides a stable and extensible interface to manage RHEL network capabilities. You can use the nmstatectl utility or the libnmstate Python library to interact with the Nmstate API.
• DHCP and Static IP Configuration: NetworkManager supports both dynamic IP address assignment using DHCP and static IP configuration. You can customize the DHCP behavior and configure specific IP addresses, gateways, and DNS settings for individual connections.

Advanced Network Configurations

Beyond basic Ethernet connections, NetworkManager in RHEL9 supports several advanced networking features:

• Network Bonding: Combine multiple network interfaces into a bond, offering higher throughput or redundancy. You can configure bonding using various tools, including nmcli, RHEL web console, nmtui, nm-connection-editor, nmstatectl, and RHEL system roles.
• NIC Teaming (Deprecated): While NIC teaming is deprecated in RHEL9, existing configurations can be migrated to network bonds using the team2bond utility.
• VLAN Tagging: Create logical networks (VLANs) within a physical network, tagging packets for traffic separation. VLAN configuration can be done using nmcli, RHEL web console, nmtui, nm-connection-editor, nmstatectl, and RHEL system roles.
• Network Bridging: Create a bridge to forward traffic between networks based on MAC addresses. Bridge configurations can be managed through nmcli, the RHEL web console, nmtui, nm-connection-editor, nmstatectl, and RHEL system roles.
• VPN Connections: Establish secure VPN connections using technologies like IPsec and WireGuard. RHEL9 offers configuration options for both through various tools and methods.

Troubleshooting and Debugging

RHEL9 provides several resources for troubleshooting and debugging NetworkManager:

• NetworkManager Logs: Examine logs for insights into NetworkManager’s operations, using the journalctl command.
• Debugging Levels and Domains: Fine-tune the verbosity of NetworkManager logs by adjusting log levels and domains in the /etc/NetworkManager/NetworkManager.conf file.
• LLDP: Utilize the Link Layer Discovery Protocol (LLDP) to detect network configuration issues, particularly incorrect VLAN configurations.
• Traffic Control: Use tools like tc and tc-ctinfo to shape network traffic and manage bandwidth rates.

An Introduction to Network Connection Profiles

NetworkManager in RHEL9 employs connection profiles to manage and store network configurations. These profiles act as containers for various settings associated with a specific network connection.

What is a Network Connection Profile?

A Network Connection Profile is a collection of parameters and values that define how a network device connects to a network. This profile encompasses details such as:

• Network Interface: This specifies the physical or virtual network interface (like enp1s0, bond0, or vlan10) associated with the connection.
• Connection Type: This defines the type of network connection (e.g., Ethernet, bond, team, VLAN, bridge, VPN, or Wi-Fi).
• IP Address Settings: This includes information about IP address assignment methods (DHCP or static), IP addresses, subnet masks, default gateways, and DNS server addresses.
• Authentication Details: For secure networks, profiles may contain credentials, certificates, and authentication protocols (like 802.1X for network authentication).
• Advanced Settings: Profiles can include advanced settings for features like VLAN tagging, bonding options, bridge configuration, VPN parameters, and quality of service (QoS) settings.

Connection Profile Formats

NetworkManager supports two main formats for connection profiles:

• Keyfile Format: This is the recommended format for RHEL9 and uses a human-readable, INI-like structure. It offers flexibility and is well-suited for managing complex configurations.
• ifcfg Format: This older, deprecated format uses separate configuration files for each interface. While still supported for compatibility, migration to the keyfile format is encouraged.

Managing Connection Profiles

RHEL9 provides multiple methods for creating, modifying, and managing connection profiles:

• Command-line (nmcli): nmcli offers a powerful and versatile way to interact with NetworkManager, including creating, editing, activating, and deactivating connection profiles.
• Graphical Tools (nm-connection-editor, GNOME Settings): For users who prefer visual interfaces, these tools provide user-friendly ways to manage profiles.
• Text-based UI (nmtui): This tool offers a terminal-based interface for managing profiles on systems without a graphical environment.
• Nmstate API (nmstatectl): This declarative API enables you to define the desired state of the network configuration, and NetworkManager ensures that the system matches that state.
• Ansible Playbooks (Network RHEL System Role): Ansible allows for the automation of connection profile management across multiple hosts using the network RHEL System Role.

Importance of Connection Profiles

Connection profiles simplify network management by:

• Storing Network Configurations: This ensures that network settings persist across system reboots.
• Enabling Easy Switching Between Networks: Users can easily switch between different networks by activating or deactivating specific profiles.
• Automating Network Configuration: Profiles can be configured to activate automatically based on various criteria, such as device availability or network SSID.

Managing and Debugging NetworkManager in RHEL

This guide provides a practical walkthrough of essential tasks and tools for managing and debugging NetworkManager in RHEL.

Customizing Network Interface Names

While RHEL9’s predictable network interface naming scheme enhances clarity, you might need to assign user-defined names for specific scenarios. 

Step 1: Identify the Target Interface’s MAC Address

Execute the following command to list network interfaces and their associated MAC addresses:

# ip link show

Note the MAC address of the interface you want to rename.

Step 2: Create the udev Rules Directory

Ensure the directory for udev rules exists:

# mkdir -p /etc/systemd/network/

Step 3: Define a udev Rule

Create a rule file (e.g., /etc/systemd/network/10-my-custom-name.rules) with the following content, replacing "00:00:5e:00:53:1a" with the target interface’s MAC address and "my_custom_name" with your desired name:

SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:00:5e:00:53:1a", NAME="my_custom_name"

This rule instructs udev to rename the interface matching the specified MAC address to the custom name.

Step 4: Apply the Changes

Reload the udev rules and restart NetworkManager:

# udevadm control --reload-rules
# systemctl restart NetworkManager

Verification: Use ip link show to confirm the interface now bears the custom name.

Alternative Naming Schemes:

• RHEL also allows switching to different network interface naming schemes.
• The default scheme, based on firmware, topology, and location, provides predictable names.
• You can explore other schemes or revert to the traditional ethX naming if needed.

Key Considerations:

• User-defined names must consist of ASCII characters and be less than 128 characters long.
• Thoroughly test the impact of custom names on other network configurations and services to avoid unintended consequences.

Troubleshooting Network Configuration Using LLDP

Link Layer Discovery Protocol (LLDP) is a valuable tool for diagnosing network configuration issues. It allows devices to share information about themselves and their neighbors, providing insights into the network topology.

Scenario: You suspect a misconfigured VLAN.

Step 1: Enable LLDP

Ensure LLDP is enabled on your RHEL host and the connected network switch.

Step 2: Gather LLDP Information

Use tools like lldpctl or the nmcli command to retrieve LLDP data. For example:

# nmcli dev show enp1s0 | grep lldp

Step 3: Analyze the Output

Examine the LLDP data for discrepancies between your host’s configuration and the switch’s settings. Focus on VLAN information. For example:

- type: 127
        ieee-802-1-vlans:
        - name: v2-0488-03-0505
          vid: 488

This output snippet reveals that the switch port associated with enp1s0 expects a VLAN ID of 488.

Step 4: Rectify Configuration Errors

If the VLAN ID on your host differs, modify your connection profile to align with the switch’s configuration using tools like nmcli or nm-connection-editor.

Key Insights:

• LLDP provides real-time visibility into the network configuration as seen by the connected switch, aiding in the identification of mismatches.
• LLDP can reveal a wealth of information beyond VLANs, including device capabilities, port descriptions, and system names.
• Familiarize yourself with the interpretation of LLDP data to effectively troubleshoot various network issues.

Analyzing NetworkManager Logs

NetworkManager’s logging system offers a valuable resource for debugging issues. The logs capture events and actions, providing clues for diagnosing problems.

Step 1: Access NetworkManager Logs

NetworkManager logs are integrated with the systemd journal. You can use journalctl to view them:

# journalctl -u NetworkManager

Step 2: Adjust Log Level and Domain

Increase the log level or specify a domain to obtain more detailed information. Modify the /etc/NetworkManager/NetworkManager.conf file and adjust the [logging] section. For instance:

[logging]
level=DEBUG
domains=ALL

This configuration sets the log level to DEBUG and enables logging for all domains.

Step 3: Restart NetworkManager

Apply the changes by restarting NetworkManager:

# systemctl restart NetworkManager

Step 4: Analyze Log Output

Carefully inspect the log output for error messages, warnings, or patterns related to the problem you’re troubleshooting. The increased logging level should provide more comprehensive details.

Key Points:

• Understanding the different log levels and domains allows you to tailor the log output to your specific needs.
• Regularly checking NetworkManager logs can help you proactively identify and address potential network issues.
• Remember to revert the log level to a less verbose setting after troubleshooting to avoid excessive log growth.