My own distorted point of view of the major issues:
- High percentage of residents and visitors from other parts of the country, and other countries, where "things work differently", and that prevents a consistent, dominant driving style (as you see in, say, Boston, which has high turnover due to the colleges, but still predominantly New England/East Coast-sourced).
- Really bad infrastructure – sometimes the "fast lane" is the one without potholes. 😉
- Really bad "road spaghetti" – lanes added, removed, under construction, left-side exits and ramps, many highway splits with little advance warning, etc… People tend to get into the lane(s) that will be split off the way they're headed, and a lot of the time (say, heading into SF from Oakland on the I-880), those will be the left lanes.
- Mountains – building large highways in this terrain is cost-prohibitive, so a lot of Bay Area highways are two lanes (or even one) in each direction where the topological maps get bumpy. Of course, one lane each way means no fast lane at all – but given the population and number of daily drivers commuting an hour or more each way to/from work, even the two lane highways are basically now "without a fast lane", simply due to traffic loads and congestion.
To the querent, I sympathize. I moved to the SFBA from Boston, land of unified and total aggression on the roads. Worse, I was born and raised in Minnesota, which (until recently, I guess) had some of the best maintained highways in the country, in large part due to the winters – you either pay to keep things clean, or you pay more to re-pave. Out here, it's the "Wild Worst" – the roads are a death sentence for your vehicle's suspension, and the people are a death sentence for you and the rest of your car. 😉