I've been working with Linux since 1999 and work with VPS servers every day, so configuring your server is no problem.
Over the last 3 years or so most of the projects I've worked on have been GPS tracking sites written in Java, and hosted on VPS systems. Currently I provide support for 12 customers with this type of system / site on VPS servers.
You probably want other things configured in addition to the email, for instance Apache, MySQL, OpenSSL, firewall, etc... Unlike shared hosting a VPS server does not have anything configured when you order the service, so there are a good half dozen or so packages that need to be installed, and configured.
If your domain name registrar or hosting provider has separate email available I suggest you go with that for incoming email, as they typically have web based email which is much easier to administer than the mail system on a VPS. Of course I will configure the VPS for outgoing email, i.e. forms on a website.
I have a couple questions:
1) Who is you VPS provider?
2) Who is the domain name registrar?
You may think my price is high, but this will take the better part of a day to configure and test everything. I have allowed for enough time to accommodate testing, and any lag in communications.
Please don't hesitate to contact me on the Freelancer message board if you have any questions or further requirements.
Best regards,
John H. Ward, Jr.