UnicodeDecodeError: ‘charmap’ codec can’t decode byte 0x81 in position 55: character maps to

I am new to Python and am hoping that someone could please explain to me what the error message means.

To be specific, I have some code of Python and SPSS combined together saved in Atom, which was created by a former colleague. Now since the former colleague is not here anymore, I need to run the code now. What I did was I ran the code below from SPSS22.

    begin program.
    import spss,spssaux,imp
    abcvalid = imp.load_source('abcvalid', "I:/VALIDITY CHECK/Python Library/2016/abcvalid2016.py") 
    import abcvalid
    abcvalid.fullprocess("9_26_2016","M:/Users/Yli2016 SURVEY/DOWNLOADS/9_26_2016/","M:/Users/Yli/2016 SURVEY/Legacy15.sav")
    end program.

Then I got the following from the output.

    Traceback (most recent call last):
      File "<string>", line 5, in <module>
      File "I:/VALIDITY CHECK/Python Library/2016/abcnvalid2016.py", line 2067, in fullprocess
        dataprep(date,filepath,legacypath)
      File "I:/VALIDITY CHECK/Python Library/2016/abcvalid2016.py", line 2006, in dataprep
        emailslower(date,filepath)
      File "I:/VALIDITY CHECK/Python Library/2016/abcvalid2016.py", line 1635, in emailslower
        DATASET ACTIVATE comment_data.""".format(date,filepath))
      File "C:PROGRA~1IBMSPSSSTATIS~122PythonLibsite-packagesspssspss.py", line 1494, in Submit
        cmdList = spssutil.CheckStr(cmdList)
      File "C:PROGRA~1IBMSPSSSTATIS~122PythonLibsite-packagesspssspssutil.py", line 166, in CheckStr
        s1 = unicode(mystr,locale.getlocale(locale.LC_CTYPE)[1])
      File "C:Program FilesIBMSPSSStatistics22Pythonlibencodingscp1252.py", line 15, in decode
        return codecs.charmap_decode(input,errors,decoding_table)
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 55: character maps to <undefined>

I know there are similar questions on this site, but the questions and answers were too hard for me to comprehend. If someone could please help me, I’d really appreciate it!

Thank you in advance!

Answer

It’s hard to be sure about what is going on here as there is a lot of code off stage, but the error message is telling you that there is an invalid character in the input stream. Code x81 is undefined in code page 1252, which is the code page in effect. That’s the western Europe/US default code page. The program is trying to convert a presumed code-page string to Unicode, so that fails.

My guess is that the input is actually not encoded with cp 1252. Something is messed up in in the Statistics current code page or with Unicode mode. You might need to set the SPSS Statistics locale to something different or to turn Unicode mode on or off. See SET LOCALE and SET UNICODE in the Command Syntax Reference on how to do this.

If you can say more about your locale and what this code is doing, we might be able to provide more information.