Darlington Chigozie Okeke -How Malware Behaves in VMs: Detection Using Heuristics & Machine Learning

Описание: Virtualisation is both a defensive advantage and an attack surface. While virtual machines provide isolation, modern malware actively detects virtualised environments, alters execution paths, delays payloads, and evades traditional endpoint security controls. This creates a challenge for infrastructure teams operating large VMware estates: how do you detect malicious behaviour that only reveals itself at runtime inside a VM?

In this session, I present a deep technical analysis of a heuristic-based machine learning framework for Trojan malware detection, developed as part of my MSc Cyber Security research and examined specifically through the lens of virtualised Windows environments.

The talk explores how malware behaves inside VMs and how that behaviour can be observed through execution artefacts such as PE structure anomalies, entropy shifts, API call sequences, DLL loading behaviour, and runtime telemetry. We examine how static and dynamic signals differ in virtualised versus bare-metal systems, and why hybrid detection approaches are required.

Attendees will gain insight into:

How malware fingerprints virtual machines and sandboxes

Which behavioural features remain reliable in VM-based execution

Engineering feature pipelines from VM-level execution telemetry

Applying Random Forest, XGBoost, and ensemble classifiers to detect Trojan activity

Detection accuracy trade-offs, false positives, and evasion techniques

Rather than focusing on reverse engineering, this session frames malware detection as an infrastructure-aware analytics problem, relevant to VMware administrators, architects, and security teams responsible for protecting virtualised workloads at scale.

This session is derived from my MSc research into heuristic and ML-based Trojan detection, but it is intentionally adapted for a VMware and virtualisation-focused audience. The emphasis is on malware behaviour within virtual machines, execution artefacts observable in VM environments, and detection trade-offs relevant to infrastructure teams.

The session is technical but does not require prior malware analysis experience. It is suitable for VMUG members interested in security, monitoring, and protecting virtualised Windows workloads.

#VMUG #LonVMUG #VMware #vCommunity #VMwareUserGroup