afxwin.h is a header file part of the MFC library, which is why it is included by the PluginMFCWindows interface. You will need to install the MFC libraries from the Visual Studio installer to use it. You can also not use MFC and implement the WindowProc using the Win32 C API.
Fun side-story: according to the Wikipedia page, AFX (for Application Framework Extensions) was the original name of the MFC framework, but it was too late to change all the headers by the time they decided on the MFC name. This is also why the PCH (pre-compiled header) for MFC projects is called "stdafx.h"!