How to highlight all duplicate rows except NaN?

I have a dataframe like this

Date ID_1 ID_2 Name
2021-2-3 12 23 Sam
2021-2-3 12 56 Sem
2021-2-3 33 56 Tom
2021-2-3 35 45 Jon
2021-2-3 35 23 Sam
2021-2-3 Nan 99 Jim
2021-2-3 35 Nan Jon
2021-2-3 88 44 Sam

I want to highlight all the duplicate rows green with just the same IDs

Date ID_1 ID_2 Name Hightlight
2021-2-3 12 23 Sam True
2021-2-3 12 56 Sem True
2021-2-3 33 56 Tom True
2021-2-3 35 45 Jon True
2021-2-3 35 23 Sam True
2021-2-3 Nan 99 Jim
2021-2-3 35 Nan Jon True
2021-2-3 88 44 Sam

Answer

You can use df.style with a custom function:

def duplicated(s):
    if s.name.startswith('ID'):
        return (s.duplicated()&s.notna()).map({True: 'background-color:green', False: None})
    else:
        return [None]*len(s)

df.style.apply(duplicated, axis=0)

Or better:

def duplicated(s):
    return (s.duplicated()&s.notna()).map({True: 'background-color:green', False: None})
df.style.apply(duplicated, subset=df.columns.str.startswith('ID'), axis=0)

highlight

Edit: for fun, here is a way to highlight the initial duplicate with a different color:

def duplicated(s):
    return ((s.duplicated().astype(int)
            +s.duplicated(keep=False).astype(int)
            )*s.notna().astype(int)
            ).map({2: 'background-color:green;color:white',
                   1: 'background-color:lightgreen',
                   0: None})
df.style.apply(duplicated, subset=df.columns.str.startswith('ID'), axis=0)

two color highlight