Installer: Cannot find python in path environment variable directly after installing it as 3rd party software

For my software (for Windows only) I create an installer with NSIS. In order for the software to work properly I deliver python (version 3.6.5) as a 3rd party software (.exe file). Further I need to conduct a pip upgrade and install some modules (e.g. NumPy).

My approach for the NSIS installer is as follows:

  1. Run executable python installer: in my installer the user is reminded to add python to the environment %path% variable. After extracting all files the 3rd party installer for python is executed. Python is installed and added to %path%.

  2. Run small C++ script (PythonSetup.exe): (Among other things) the script matches the path variable for the python directory using GetEnvironmentVariable from Windows.h. If python is in path then it executes some commands like

python -m pip install --upgrade pip
python -m pip install numpy

otherwise it just prints “python not found” and the whole path environment variable.

In NSIS it is implemented like this (simplified):

SetOutPath "$INSTDIRpython"
File "D:pythonpython-3.6.5.exe"
ExecWait '"$INSTDIRpythonpython-3.6.5.exe" InstallAllUsers=1 PrependPath=1'
Sleep 500
File "D:pythonPythonSetup.exe"
ExecWait '"$INSTDIRpythonPythonSetup.exe" $INSTDIR'
Sleep 500
SetOutPath "$INSTDIR"
Delete "$INSTDIRpythonpython-3.6.5.exe"
Delete "$INSTDIRpythonPythonSetup.exe"
RmDir "$INSTDIRpython"

My problems are:

  • PythonSetup.exe cannot find the python directory in the %path% variable. Printing the path I got using GetEnvironmentVariable from Windows.h, I can see that the path it gets really does not contain the python directory.
  • Forcing the commands metioned above I get the error: 'python.exe' is not recognized as an internal or external command, operable program or batch file.
  • However checking the %path% variable in Windows System Properties > Advanced > Environment Variables shows that the python directory has indeed been added to the path variable. I checked this during the runtime of PyhtonSetup.exe, i.e. during the NSIS line ExecWait '"$INSTDIRpythonPythonSetup.exe" $INSTDIR'.
  • Executing PythonSetup.exe immediately after the NSIS installer is finished works just fine.

I tried

  • a longer Sleep
  • other python architectures (32bit vs 64bit)
  • looking for the python directory manually and calling the python.exe that way, e.g.
C:Program Filespython36-32python.exe -m pip install numpy

Does anyone know the reason why the python directory cannot be found in the path that is returned by GetEnvironmentVariable in the PythonSetup.exe while at the same time it can be seen in the System Properties? I feel like I’m probably missing something obvious…

Answer

When a user logs on Windows reads the environment variables from the User and Machine parts of the registry and starts the initial process with these merged variables. When a new process is created it inherits the environment of its parent (unless you specify a new custom environment when starting the process).

Applications are supposed to broadcast the WM_SETTINGCHANGE message when they change the environment but in practice Explorer is the only application that listens for this message and dynamically updates its environment. A major reason for this is that Explorer uses a undocumented function to perform this update and it is a major pain for other applications to implement the same functionality.

For applications that uses custom variables it is easy to set this value in the installer so it is inherited by child processes it starts:

System::Call 'KERNEL32::SetEnvironmentVariable(t "MYAPP_DIR", t "$InstDir")'

Updating %PATH% the same way is tricky for various reasons:

  • NSIS has a string length limit and a long %PATH% might not fit.
  • %PATH% is a combination of values from HKCU and HKLM and requires a bit of string manipulation to join them correctly.

My only suggestion for updating %PATH% in the installer is to use the EnVar plug-in:

EnVar::Update "" "PATH"
Pop $0 ; "0" on success