If you couldn't get through Portrait (which is great yet pretty accessible) you'll never make it through the first chapter (episode) of Ulysses.
Think about the two books as companion pieces: Portrait is about a young artist getting his wings, and at the end he takes off. Ulysses begins with his sudden crash. There's an Icarus metaphor in there, but of course Ulysses was modelled (loosely) after Homer's Odyssey (about as much as the Coen Bros. film "O Brother, Where Art Thou?" was).
But Ulysses is a comedy. You just have to have the history of western literary tradition in your back pocket to "get" much of it. However, even if you just pick up the obvious stuff, the plays on words, the poon-hound Leopold Bloom or the full of himself young twit of Stephen Dedalus (Joyce's alter-ego of sorts) you'll have a good read.
But it's not really something you'd bring to the beach, unless you're a geek like me.
And if you want a decent version of Ulysses, you have to read the one edited by Hans Gabler. It *does* too matter.
-- Jough