HACKER Q&A
📣 houseinthewoods

How do you inspect JSON logs?


TLDR if you don't want to stick around for my ramble: if you have experience inspecting JSON logs, what's your workflow?

I'm working on a project of a very small scale where monitoring is important. I'm biased to just stick with text files and SSH rather than log software (such as ELK stack or Grafana Loki), so as to not have to learn/install/maintain a heavy tool for my toy needs. I'm also biased towards JSON as a format because I learned programming too late in life to be a Unixy regexy text wizard, though I admire people of that ilk from afar.

For improvised analysis, I'm using jq on the terminal. For less flexible human-readable text summaries, I wrote a Deno script. Ergonomically, jq on the terminal feels lacking. Whoever made jq is doing God's work, to be clear, but I could possibly use a more intuitive language (with an emphasis on more tersely expressing groupings/filters). And independently of using jq or not, the terminal is slow and ephemeral: I would love something sort of like the Acme text editor (where you can build yourself an interface on the fly with text), but without the three-button mouse dependency. Right now I just keep a cheatsheet of jq commands that have been useful. I've been daydreaming about making a local webapp to fulfill this need (that just eval()s Javascript, since it would only ever be local).


  👤 sethammons Accepted Answer ✓
The big player here is Splunk. Aside from a free trial, $40/mo minimum I think for their cloud offering. They get very pricey. I loved using it at my last company. You can search and find relationships in the logs, add monitoring and graphs and reports and and and ....

👤 yathrowaway2
I think LogQL might be the language you seek; I'd try using Loki, the setup might not be as heavyweight as you think.

See for example something like: https://blog.ruanbekker.com/blog/2020/08/13/getting-started-...

You can run a couple of docker containers (loki + grafana), import the data you need (a bit of promtail config). That's not saving anything inside of that stack, you'll want to save your work in Grafana, but for loki maybe you don't need to save much, just import what you need via promtail.


👤 java-man
A similar question from 2018:

https://news.ycombinator.com/item?id=17424627


👤 java-man
Could you describe your most common use case(s) please?

👤 account-5
This is off topic but if it's log files and you want simple processing you couldn't go wrong with awk.

Obviously I don't know if the output is default JSON and not a traditional log file output, that is transformed into JSON.

If your learning jq on the terminal awk is really easy.


👤 cranberryturkey
you need a ndjson parser. i saw one just the other day but I can't remember what it was called.