How can I highlight categorical variables in pandas dataframe in Python?

I have a pandas dataframe called value_matrix_classification which looks as follows:

{('wind_on_share',
  'Wind-onshore power generation'): {('AIM/CGE 2.0',
   'ADVANCE_2020_WB2C'): 'high', ('AIM/CGE 2.0',
   'ADVANCE_2030_Price1.5C'): 'high', ('AIM/CGE 2.0',
   'ADVANCE_2030_WB2C'): 'high', ('IMAGE 3.0.1',
   'ADVANCE_2020_WB2C'): 'low', ('IMAGE 3.0.1',
   'ADVANCE_2030_WB2C'): 'low', ('MESSAGE-GLOBIOM 1.0',
   'ADVANCE_2020_WB2C'): 'low'},
 ('wind_off_share',
  'Wind-offshore power generation'): {('AIM/CGE 2.0',
   'ADVANCE_2020_WB2C'): nan, ('AIM/CGE 2.0',
   'ADVANCE_2030_Price1.5C'): nan, ('AIM/CGE 2.0',
   'ADVANCE_2030_WB2C'): nan, ('IMAGE 3.0.1',
   'ADVANCE_2020_WB2C'): 'low', ('IMAGE 3.0.1',
   'ADVANCE_2030_WB2C'): 'low', ('MESSAGE-GLOBIOM 1.0',
   'ADVANCE_2020_WB2C'): 'low'}}

The two columns in the right contain low, medium and high which are categorical variables. I created them using pd.cut(value_matrix_classification, bins = 3, labels = ["low", "medium", "high"]

I’d like to highlight the pandas dataframe such that there are red, orange, yellow and background color for high, medium, low and NaN values respectively.

I wrote the following function

def highlight_cells(x):
    if x == "high":
        color = "red"
    elif x=="medium":
        color = "orange"
    elif x=="low":
        color = "yellow"
    else:
        color = "gray"
    
    return [f"background-color: {color}"]

and applied it to the dataframe

value_matrix_classification.style.apply(highlight_cells)

However, this gives ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). What would be the appropriate way to do the highlighting here?

I was able to highlight the cells with null values only using

value_matrix_classification.style.highlight_null(null_color = "gray")

I am attaching the screenshot here just for the convenience of the reader. enter image description here

How can I highlight all the cells based on the given categories: low, medium and high?

Answer

apply takes an entire row or column as input. Use applymap instead.

See this Pandas documentation section.

Edit: you’ll also want highlight_cells to return just f"background-color: {color}", not wrapped in a list.