JSON Command Line Tools: Mastering jq for JSON Processing

jq is a lightweight, powerful command-line JSON processor. Think of it as sed/awk for JSON — it slices, filters, maps, and transforms JSON data with a concise functional query language. Every developer who works with JSON on the command line should know jq. This guide covers installation, basic queries, filters, transformations, and real-world use cases. Use our JSON Formatter for visual inspection before applying jq queries and JSON Validator to check jq output.

Installation

# macOS
brew install jq

# Linux (Ubuntu/Debian)
sudo apt-get install jq

# Linux (CentOS/RHEL)
sudo yum install jq

# Windows (Chocolatey)
choco install jq

# Windows (Scoop)
scoop install jq

# Verify installation
jq --version  # jq-1.7

Basic Usage

# Pretty print JSON from pipe
curl https://api.example.com/users | jq '.'

# Pretty print from file
jq '.' data.json

# The '.' filter outputs the input unchanged (identity)
# With jq default pretty printing (2-space indent + colors)

# Disable color output
jq -M '.' data.json

# Compact output (minified)
jq -c '.' data.json

Essential Filters Reference

Filter	Description	Example
.	Identity (output whole input)	jq '.'
.key	Access object property	jq '.name'
.key1.key2	Nested property access	jq '.address.city'
.[]	Iterate over array elements	jq '.[]'
.[0]	Access array by index	jq '.[0]'
.[-1]	Last array element	jq '.[-1]'
.key[]?	Optional (no error if missing)	jq '.tags[]?'
select(.key == val)	Filter objects by condition	jq '.[] | select(.age > 18)'
map(.key)	Transform array elements	jq 'map({name: .name})'
group_by(.key)	Group array by field	jq 'group_by(.country)'
sort_by(.key)	Sort array by field	jq 'sort_by(.name)'
length	String or array length	jq '.[] | length'
keys	Object key names	jq 'keys'
add	Sum array of numbers	jq '[.[].age] | add'
unique	Deduplicate array	jq '[.[].city] | unique'

Real-World Examples

// Sample data: users.json
[
  {"id": 1, "name": "Alice", "age": 30, "city": "NYC"},
  {"id": 2, "name": "Bob", "age": 25, "city": "SF"},
  {"id": 3, "name": "Charlie", "age": 35, "city": "NYC"}
]

# Extract all names
jq '.[].name' users.json
# "Alice"
# "Bob"
# "Charlie"

# Filter: users older than 28, only name and age
jq '.[] | select(.age > 28) | {name, age}' users.json
# { "name": "Alice", "age": 30 }
# { "name": "Charlie", "age": 35 }

# Group by city
jq 'group_by(.city) | map({city: .[0].city, count: length, users: [.[].name]})' users.json
# [
#   { "city": "NYC", "count": 2, "users": ["Alice", "Charlie"] },
#   { "city": "SF", "count": 1, "users": ["Bob"] }
# ]

# Compute average age
jq '[.[].age] | add / length' users.json
# 30

# Transform to key-value object
jq 'map({(.name): .age}) | add' users.json
# { "Alice": 30, "Bob": 25, "Charlie": 35 }

Advanced jq Features

# Raw string output (no quotes)
jq -r '.[].name' users.json

# Custom output format (raw strings)
jq -r '.[] | "(.name): (.age)"' users.json
# Alice: 30
# Bob: 25

# Build complex objects
jq '{ total_users: length, average_age: ([.[].age] | add / length), cities: [.[].city] | unique }' users.json

# Using variables
jq --arg min_age 28 '.[] | select(.age >= ($min_age | tonumber))' users.json

# Slurp input (read entire input as array)
jq -s '.' file1.json file2.json

# Merge objects
jq -s '.[0] * .[1]' base.json override.json

# Recursive descent (find all 'name' keys at any depth)
jq '[.. | objects | select(has("name")) | .name]' data.json

Common jq Pipelines

# API debugging
curl -s https://api.example.com/users | jq '{ count: length, data: .[0:3] }'

# Log analysis
cat access.log | jq -r 'select(.status >= 400) | "(.timestamp) (.method) (.path) -> (.status)"'

# Kubernetes (get pod names)
kubectl get pods -o json | jq '.items[].metadata.name'

# Docker (get running container IDs)
docker ps --format '{{json .}}' | jq -s '.[].ID'

# Terraform state query
jq '.resources[] | select(.type == "aws_instance") | .instances[].attributes.id' terraform.tfstate

# Package.json scripts
jq '.scripts | to_entries[] | "(.key): (.value)"' package.json

Next Steps

Format JSON before querying with JSON Formatter. Validate jq output with JSON Validator. For visual exploration, use JSON Tree Viewer.