We know games are all about fun. And the greatest question that all game designers try to solve is: "What is fun?". It's an impossible question, because everybody is different in many ways. However, we must also realize that everybody is the same in many ways too. The answer to 'What is fun?' is actually not totally impossible.
So how to we answer the 'What is fun' question? The other way to look at the question is 'What isn't fun'? Now that question suddenly seems more answerable. I feel that it's because people remember shitty things more than good things. Human are first pessimistic before they become optimistic. They look at shitty things around them and try to improve it, either by effort, delusion, offsetting pros and cons, etc. Only then they become optimists. This is one of the things I observed in the army.
The main idea is, we usually realize more bad things than good. When the bad starts to outweigh the good, we become frustrated. In fact, as long as we have too many bad things in the game design, we get frustrated.
Frustration exists in so many areas in a video game. Loading screens, unwanted cause and effects, lack of goals and objectives, waiting times, fairness, difficulty, ugly graphics and art design, bad camera angles, the list goes on and on and on. In fact, there are ways to frustrate players that you can't really list in just phrases.
So as you can see, the question isn't really "What is fun" first. For a game to become fun, first it must eliminate what isn't fun. By looking at this perspective, I believe, you will be able to keep track of the bigger picture of your game. By doing so, you will be able to polish your game. You will start to know why companies like Blizzard takes such a damn long time to release their incredibly polished games. There's always something to be done.
Because what (you believe) is fun usually revolves in the gameplay, I find that sometimes designers overlook other smaller aspects of the game like UI, music, sound, menu user-bility, instructions, and other accessibility issues.
If you look at board games, vanilla Catan is fun for the first or second playthrough, until you realize how dependent on luck it is. People will start to slowly become frustrated when they didn't get their stuff when they logically should. Finally when they realize that they are being controlled by circumstances, they quit.
A comic strip inspired by Settlers of Catan's gameplay
Another example, this time a video game, is 'Cross Edge' on PS3. I played through the game and gave it like tons of chances. I know I'm not the most tolerant of gamers, but sometimes almost every aspect of the game turns me off. The music is not fitting, the gameplay is novel but tedious, difficult and has a ridiculously high learning curve, and the controls and mechanics are quite confusing.
More painful to play than it looks
If I weren't a JRPG player, I would've dropped it the first 10 mins into the game. Of course, I'm sure people still play it, but if you look at the package, you know it's going to be, at most, a cult hit. And it's going to be popular only because of its characters.
Sometimes I wonder what some of these devs are thinking. Are they just making games just to hope that fans will love it just because of its popular characters? I understand that some sources of frustration are beyond the game developers' control (like players dying constantly in games like DOTA due to lack of experience, stupidity or bad teammates), but there are sources which are totally controllable.
Everytime I play a game, I would always think what the designers were thinking. Why did they do little things like this and that, from gameplay, to UI, to sound. Sometimes I find some parts frustrating and wonder why the developers overlooked it. Either they honestly did not notice, there was no time to fix it, or they don't give a damn. 'Cross Edge' truly felt like the devs don't even care.
Of course, nothing can be compared to the all famous Big Rigs:
Before you start making a game, video or board game, you must first decide its audience. There is a reason why Catan is designed that way and I believe it's not really for the hardcore people who enjoy scheming a planning like in a chess game. The only reason why Catan was designed that way, I believe, was because it's catered to casual players. If you look at Catan that way, it actually makes a lot of sense.
And then there are the grey areas; aspects of the game where you really need to know how much your target audience tolerate. You have to find out what your audience is able to tolerate. This is especially true for a game that is heavily based on numbers. Not everyone 'gets' Civilization series or the like as the numbers probably overwhelm them. In such games, aspects that could probably be labeled as 'frustrating' to most would become a 'necessity' for the target audience.
However, even if an aspect is necessary, it is still important to make it as un-frustrating as possible. Not everything is perfect, but it can be close to being one.