The system call vm86 was introduced in Linux 0.97p2. In Linux 2.1.15 and 2.0.28 it was renamed to vm86old, and a new vm86 was introduced. The definition of struct vm86_struct was changed in 1.1.8 and 1.1.9.
These calls cause the process to enter VM86 mode (virtual-8086 in Intel literature), and are used by dosemu.
VM86 mode is an emulation of real mode within a protected mode task.
On success, zero is returned. On error, -1 is returned, and errno is set appropriately.