Python3: Merging multiple dictionaries into a new dictionary

I apologize if this is a duplicate. I’m sure it is, I think I just don’t know how to frame the question I’m trying to ask to be able to search for the correct answers.

I have a function that returns a dictionary per line, like so:

{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'AUDIOUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}
...
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1m', 'scenario': -2, 'continuity': 0}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '3m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '5m', 'scenario': 1, 'continuity': -1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '15m', 'scenario': 1, 'continuity': -1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '30m', 'scenario': 2, 'continuity': 1}
{'symbol_name': 'TRBUSDT', 'timeframe_name': '1h', 'scenario': 2, 'continuity': 1}

I want to create a new dictionary line that merges the data for the lines that have a common symbol_name, like so:

{'symbol_name': 'AUDIOUSDT', '1m_scenario': 1, '1m_continuity': 1, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': 1, '15m_scenario': 1, '15m_continuity': 1, '30m_scenario': 2, '3m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1 }
...
{'symbol_name': 'TRBUSDT', '1m_scenario': -2, '1m_continuity': 0, '3m_scenario': 2, '3m_continuity': 1, '5m_scenario': 1, '5m_continuity': -1, '15m_scenario': 1, '15m_continuity': -1, '30m_scenario': 2, '3m_continuity': 1, '1h_scenario': 2, '1h_continuity': 1 }

I’m having a hard time figuring out how to do this. I’ve tried to break this down into small bite-size chunks so I can get the hang of it, starting with just getting each timeframe into a new dictionary based on symbols_name:

data = {}
for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']    
    data[symbol] = {}
    data[symbol][timeframe] = {}

print(data)

# returns:
{'AUDIOUSDT': {'1h': {}}, ... 'TRBUSDT': {'1h': {}}}

In the above example, I thought I would see the other timeframe keys in the new dictionary line I’m trying there other than just 1h, because each time the loop runs, timeframe_name is different, but I just see the last timeframe_name.

I tried a different variation, but it just got me back to where I had started:

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    data = {}
    data[symbol] = {}
    data[symbol][timeframe] = {}
    print(data)

# returns:
{'AUDIOUSDT': {'1m': {}}}
{'AUDIOUSDT': {'3m': {}}}
{'AUDIOUSDT': {'5m': {}}}
{'AUDIOUSDT': {'15m': {}}}
{'AUDIOUSDT': {'30m': {}}}
{'AUDIOUSDT': {'1h': {}}}
...
{'TRBUSDT': {'1m': {}}}
{'TRBUSDT': {'3m': {}}}
{'TRBUSDT': {'5m': {}}}
{'TRBUSDT': {'15m': {}}}
{'TRBUSDT': {'30m': {}}}
{'TRBUSDT': {'1h': {}}}

Any pointers?

Answer

You can use setdefault and update methods of dict:

data = {}

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    d = data.setdefault(symbol, {})
    d.update({'symbol_name': symbol,
              f'{timeframe}_scenario': item['scenario'],
              f'{timeframe}_continuity': item['continuity']})
>>> list(data.values())
[{'symbol_name': 'AUDIOUSDT',
  '1m_scenario': 1,
  '1m_continuity': 1,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': 1,
  '15m_scenario': 1,
  '15m_continuity': 1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1},
 {'symbol_name': 'TRBUSDT',
  '1m_scenario': -2,
  '1m_continuity': 0,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': -1,
  '15m_scenario': 1,
  '15m_continuity': -1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1}]

Old answer (to avoid duplicate answer with @AkshaySehgal)

Use defaultdict from collections module

from collections import defaultdict

data = collections.defaultdict(dict)

for item in query:
    symbol = item['symbol_name']
    timeframe = item['timeframe_name']
    data[symbol].update({'symbol_name': symbol,
                      f'{timeframe}_scenario': item['scenario'],
                      f'{timeframe}_continuity': item['continuity']})

Output:

>>> list(data.values())
[{'symbol_name': 'AUDIOUSDT',
  '1m_scenario': 1,
  '1m_continuity': 1,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': 1,
  '15m_scenario': 1,
  '15m_continuity': 1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1},
 {'symbol_name': 'TRBUSDT',
  '1m_scenario': -2,
  '1m_continuity': 0,
  '3m_scenario': 2,
  '3m_continuity': 1,
  '5m_scenario': 1,
  '5m_continuity': -1,
  '15m_scenario': 1,
  '15m_continuity': -1,
  '30m_scenario': 2,
  '30m_continuity': 1,
  '1h_scenario': 2,
  '1h_continuity': 1}]