Exclude or filter json keys from a json object and write to file

I’m currently using a python module that helps with the tenable API to export asset data from tenable. The export function returns an “ExportIterator” type object to walk through the results of the export.

Essentially this returns too much data per asset, and I’m having difficulty figuring out how to filter out the data being returned so I can use it.

The module shows the following function as an example to pull asset information:

assets = tio.exports.assets()
for asset in assets:
  print(asset)

This returns thousands of json objects with hundreds of keys (I’ve removed and obfuscated several) like this:

{
    "id": "1a2b3c",
    "has_plugin_results": true,
    "created_at": "xxx",
    "terminated_at": null,
    "terminated_by": null,
    "updated_at": "xxx",
    "deleted_at": null,
    "deleted_by": null,
    "first_seen": "",
    "last_seen": "",
    "first_scan_time": "xxx",
    "last_scan_time": "xxx",
    "last_authenticated_scan_date": "xxx",
    "last_licensed_scan_date": "xxx,
    "last_scan_id": "xxx,
    "last_schedule_id": "xxx",
    "azure_vm_id": null,
    "azure_resource_id": null,
    "gcp_project_id": null,
    "gcp_zone": null,
    "gcp_instance_id": null,
    "aws_ec2_instance_ami_id": null,
    "aws_ec2_instance_id": null,
    "agent_uuid": "xxx",
    "bios_uuid": "xxx",
    "network_id": "xxx",
    "network_name": "Default",
    "aws_owner_id": null,
    "aws_availability_zone": null,
    "aws_region": null,
    "aws_vpc_id": null,
    "aws_ec2_instance_group_name": null,
    "aws_ec2_instance_state_name": null,
    "aws_ec2_instance_type": null,
    "aws_subnet_id": null,
    "aws_ec2_product_code": null,
    "aws_ec2_name": null,
    "mcafee_epo_guid": "{xxx}",
    "mcafee_epo_agent_guid": "{xxx}",
    "servicenow_sysid": null,
    "agent_names": [
        "aaabbbccc123"
    ],
    "installed_software": [],
    "ipv4s": [
        "1.1.1.1",
        "2.2.2.2"
    ],
    "ipv6s": [],
    "fqdns": [
        "aaabbbbccc"
    ],
    "mac_addresses": [
        "aa:bb:cc"
    ],
    "netbios_names": [
        "aaabbbccc123"
    ],
    "operating_systems": [
        "foobar 10"
    ],
    "system_types": [
        "general-purpose"
    ],
    "hostnames": [
        "aaabbbccc123"
    ],
    "sources": [
        {
            "name": "AGENT",
            "first_seen": "xxx",
            "last_seen": "xxx"
        }
    ],
}

This module function for exporting doesn’t support any arguments for filtering the json object itself. I just want a json file that lists these objects with “hostnames”, and “ipv4s”.Here is my current code to try and filter this asset data to a json file with only the key:value elements I want:

from tenable.io import TenableIO
import json
import os

# api-key as an environment variable

tio = TenableIO()

# doc for pytenable exports function https://pytenable.readthedocs.io/en/stable/io.html#exports

wr = open('assets.json','w')

#create variable to iterate through the export function results
for asset in tio.exports.assets:
    new_data = [{ "hostnames" : x["hostnames"], "ipv4s" : x["ipv4s"]} for x in asset[{}]]
    wr.write(json.dumps((new_data), indent = 4))

wr.close()

Currently receiving:

 new_data = [{ "hostnames" : x["hostnames"], "ipv4s" : x["ipv4s"]} for x in asset[{}]]
TypeError: unhashable type: 'dict'

desired output:

{
   "hostnames": [
        "aaabbbccc123"
    ],
    "ipv4s": [
        "1.1.1.1",
        "2.2.2.2"
    ], 
}
{
   "hostnames": [
        "dddeeefff456"
    ],
    "ipv4s": [
        "1.1.1.1",
        "2.2.2.2"
    ], 
}

any ideas on how to proceed?

Answer

Asset should already be a single json object here. Json objects are translated into dictionaries in most Python json parsing libraries, so you could do soemthing like:

for asset in tio.exports.assets:
    new_data = [{ "hostnames" : asset["hostnames"], "ipv4s" : asset["ipv4s"]}]
    wr.write(json.dumps((new_data), indent = 4))

wr.close()

if not every json blob you’re receiving will have the “hostnames” and “ipv4s” keys, then I’d recommend using to following to check if the key exists before trying to access it:

if "hostnames" in asset:
    # add hostnames to the new dictionary here