I had a similar problem with my Jet, then Bernzy stopped by and showed me how to set it up right
A fence will definitely help. Sears sells a universal bandsaw fence for around $20. Works pretty well. If I need real straight cuts I'll also use a clamp on the other end of the fence. If you do not want to string for a fence, clamp a piece of lumber to the bandsaw and use that as your fence.
Jigman