If you uninstall Java in Windows using the Control Panel Add/Remove programs feature you might expect it to not leave behind any Java related files but while investigating a problem with deployJava.js auto-installation I discovered that the Java Update Scheduler (jusched.exe) is left behind and will continue to start up with Windows.
The executables can be found in (on Windows XP)
C:\Program Files\Common Files\Java\Java Update
Which contains the files
jaucheck.exe
jaureg.exe
jucheck.exe
jusched.exe
task64.xml
task.xml
Deleting these files will prevent them from auto starting and using up memory and CPU cycles but will leave behind registry entries.
The registry key which starts jusched.exe is located in
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
and the key itself is named
SunJavaUpdateSched
Removing this key will prevent Windows from searching for and failing to find jusched.exe