A cli JSON processor

Let me introduce jop, a command-line JSON processor powered by node.js

jop is able to filter, sort, transform JSON input via a pipeline of operations.

 → cat people.json
[{"name":"Andrea","age":19},{"name":"Beatrice", "age": 21},{"name":"Carlo", "age":16}]

 → jop --findall "it.age > 18" -p people.json 
[
  {
    "name": "Andrea",
    "age": 19
  },
  {
    "name": "Beatrice",
    "age": 21
  }
]

 → jop --findall "it.age > 18" --collect "name" people.json 
[ "Andrea", "Beatrice" ]

Many operations are available:

Options:
  -c, --count      Count all elements in input                                                      
  --countby        Count elements in input grouping by the given expression                         
  -f, --findall    Find all elements satisfying given expression                                    
  --find           Find first element satisfying given expression                                   
  -g, --groupby    Group input elements using given expression                                      
  --indexof        Find index of the first element matching the given expression                    
  --lastindexof    Find index of the last element matching the given expression                     
  --collect        Collect the value of the given property name the input collection                
  --min            Find the element having the max output for the given expression                  
  --max            Find the element having the min output for the given expression                  
  --head           Return the first n nodes, where n is a given number            
  -s, --sortby     Sort input elements using the optionally provided expression                     
  --sample         Get n random elments from input, where n is a given number        
  -t, --transform  Transform input node using the given expression
  --tail           Return last n elements, where n is a given number       
  -p, --pretty     Pretty-print output                                                              
  --prop           Return value for a given property name                                           
  --keys           Collect object keys from input                                                   
  --values         Collect object values from input

Some operations require a javascript expression that is evaluated on each item of the input collection to perform the requested manipulation. The symbol it is bound to the currently iterated item (eg. for each object return current item if its age property is above 18).

 → jop --findall "it.age > 18" people.json 
[{"name":"Andrea","age":19},{"name":"Beatrice", "age": 21}]

Others accept as parameter a property name (eg. collect values for the age property) or a number (eg. display the first two items of the input collection):

 → jop --collect "name" people.json 
[ "Andrea", "Beatrice", "Carlo" ]

 → jop --head 2 people.json 
[{"name":"Andrea","age":19},{"name":"Beatrice", "age": 21}]

The output of one operation is used as input by the next one (eg. collect names for people older than 18):

 → jop --findall "it.age > 18" --collect "name" people.json 
[ "Andrea", "Beatrice" ]

One of the most remarkable feature of jop is the ability to use a javascript expression to transform JSON structure in input creating a brand new collection of objects,

 → jop -t "out[key] = { name: it.name, yob: new Date().getFullYear() - it.age }" -p people.json
[
  {
    "name": "Andrea",
    "age": 19,
    "yob": 1995
  },
  {
    "name": "Beatrice",
    "age": 21,
    "yob": 1993
  },
  {
    "name": "Carlo",
    "age": 16,
    "yob": 1998
  }
]

or appending properties to the existing ones

 → jop -t "out[key] = _.merge( _.mapValues(it), {yob: (new Date().getFullYear() - it.age)} )" -p people.json
[
  {
    "name": "Andrea",
    "age": 19,
    "yob": 1995
  },
  {
    "name": "Beatrice",
    "age": 21,
    "yob": 1993
  },
  {
    "name": "Carlo",
    "age": 16,
    "yob": 1998
  }
]

The example above shows how to access, in the expression context, to the:

output object via the out
current object key via the key
Lo-Dash library methods via _

Motivation Link to heading

I am a fan of command line tools. In many situations I would have wished something like grep or awk to manipulate JSON content. I tried jq, which seems very good. However I was not quite able to grasp its syntax, so I gave up searching and I created my own.

At first I implemented the tool in Groovy. Results were quite interesting: Groovy has an amazing support for handling JSON, moreover the expressiveness of the Groovy syntax comes quite handy while writing node manipulation expressions. However JVM slow startup time is a big issue for a commandline tool like jop. So I re-engineered the tool using node.js.

Installation and usage Link to heading

For both install informations and an extensive list of examples, please refer to the project page on Github.

Feel free to contribute with pull requests and/or submitting bugs.