There is definitely a political and philosophical divide in this country, but that divide, or rather multiple divides run much deeper than the surface appearance of "Left" vs "Right", "Liberal" vs "Conservative", or "Socialism" vs "Capitalism." Most of these things that gain so much attention and receive so much focus are merely window dressing, arbitrary and meaningless labels that are used to paint a person, group or idea as "enemy" and another person, group or idea as "ally" arbitrarily, and they hold little or no real meaning or value of their own (especially the archaic and horrendously inaccurate "Left vs Right" axial concept).
From what I have seen, all political divides that aren't entirely manufactured to allow politicians to gain power and influence or to allow cheaters and abusers of the system to block and delay and hamstring steps to correct their abuses, revolve around three primary axes of philosophy. There are many more, but they are mostly minor or are sub-axes of these main clusters. The first tends to fall mostly along the artificial line defined by the popular left-vs-right axis, in no small part because it is the philosophy most used (abused) to drive that artificial axis. The other two tend to overlap with that divide as well, though in varying degrees, with the last being the most scattered. These three philosophical divides are Collectivism vs Individualism, Look-Forward Optimism vs Look-Back Nostalgia, and Rationalism and Reality-Based Thinking vs Fantasy-Based Thinking and Embracing Cognitive Dissonance.
Collectivism vs Individualism
This main philosophical divide is the one that is most easily recognizable, and the most prominent because it has been most heavily used to drive the "left vs right", "liberal vs conservative" and "socialist vs capitalist" feuds. It is also the philosophical axis that has been most heavily abused.
Under Individualism in its purest, most extreme form, only the single individual matters. Nothing else is important or of any value beyond what use it can be to the individual. The purist form of an Individualist is an Unprincipled Narcisist with Anti-Social tendencies. The sociopath who does not view anything beyond himself as important, or even real. To the radical Individualist, the universe and the self are indistinguishable, because to the radical individualist the universe’s value is determined entirely by his existence and state of existence, or that he IS the universe and the only thing that is real. Carried to its logical conclusion, it results in total anarchy with constant conflict between individuals in a continual every-man-for-himself war, OR it leads to the totalitarian rule of a singular entity with the forced oppression and obedience of the masses, with a frequently changing paranoid leadership constantly watching for a knife in the back (see every system of feudal aristocracy that has ever existed across the last 14,000 years).
Less radical Individualists view the individual person as having value in and of itself. Being a singular person makes one valuable and important, and a single life has great significance and even sacrament. The identity of the self, as distinct from others, is of prime importance, with groups the self resides within having only secondary precedence. It is from this philosophy that individual rights stem, because if the individual has no value or importance or sacrament, then the individual has no claim to any rights by its very nature. It is also from this philosophy that much of our competitive drive and energy and our individual diversity comes from, the desire of the individual to distinguish itself above or apart from the others, to be recognized and noticed individually, instead of as just a part of a group.
Under Collectivism in its purest, most extreme form, only the collective group as a whole matters. No individual has any value in and of itself, beyond what it can contribute to the collective whole, and whole swaths of individuals can be sacrificed without hesitation for the good of the collective. The Borg Collective in Star Trek (as they were portrayed before the introduction of the Borg Queen in the eighth movie, Star Trek: First Contact) is a classic example of pure Collectivism. There is no individual in such a society, the individual has no identity, just a designation as a part of the whole that is irrelevant in and of itself, beyond what service it can provide to the whole, much like the individual cells in your body (in fact, the human body and all multi-cellular life is a biological implementation of pure collectivism, with the individual cells having effectively zero value on their own, even when factoring in what they each individually contribute to the whole body). Carried to its logical conclusion, it leads to the total erasure of the individual in a totalitarian state of mob rule, with huge swaths of people exterminated at the collective whim of the group, or huge swaths of people heartlessly sacrificed for the good of the whole.
Less extreme Collectivists view the group as having more collective value than the individual; Collectivists don't necessarily reject the idea of the individual having value in and of itself, they just put more weight of value on a collection of several individuals or the entire group as a whole than on a single individual. The concepts of sharing of resources and mutual cooperation for mutual benefit stem from this philosophy, and the idea that an individual greed at the expense of others is wrong also comes from the Collectivist philosophy.
Now, Extreme Collectivism and Extreme Individualism are mutually exclusive, they cannot coexist. But moderate/sane/not-ridiculous Collectivist and Individualist philosophies can coexist quite easily. Even though it seems counter-intuitive a Collectivist would say, "The needs of the many outweigh the needs of the few or the one," and an Individualist would say, "The needs of the one outweigh the needs of the many," these statements are not mutually exclusive, and CAN both be true, even simultaneously. The life of the one outweighs the comfort of many, while the comfort of many outweighs the luxury of the one. The policies of sane Collectivists and sane individualists often don't even conflict, because often times what is good for the group as a whole also simultaneously benefits the individual, and for less individual effort. Taxes to pay for public education, for example, benefit the whole because it provides funding to increase the education level of the whole (thus increasing the collective level of intelligence, knowledge and wisdom), which also benefits the individual who can take advantage of the public education, and who will experience a better life by having a better education and being surrounded by people with a better education, making them more skilled, better able to make good, safe and successful decisions, come up with and implement new ideas, etc. In many ways, sane Collectivism and sane Individualism complement each other, and in the few areas where they conflict, sane Collectivists and sane Individualists can compromise, either accepting to follow the policy of one or the other if it works best, or creating a hybrid policy.
We see so little of this today, however, because this philosophical axis has been so abused by modern politics that play on painting opponents with the extreme version of whichever 'side' they are against, to make them out to be an evil/mad/insane Enemy to be Resisted At All Costs. So we have sane Individualists looking at sane Collectivists and seeing insane Collectivists because the only kind of Collectivism they have been trained to see is Extreme Collectivism, and you have sane Collectivists looking at sane Individualists and seeing insane Individualists because the only kind of Individualism they have been trained to see is Extreme Individualism. This in turn then drives both sides further towards their Extremes in response to the perceived Extremism on 'the other side.'
This is one of the biggest drivers of the fierce political conflict we see today, and why Capitalists hate Socialists and Socialists hate Capitalists, even though both systems can mutually coexist and greatly benefit each other. The Sane-Individualism Capitalists can only see the Sane-Collectivism Socialists as Insane-Collectivist Communists, and the Sane-Collectivism Socialists can only see the Sane-Individualism Capitalists as Insane-Individualism Feudal Aristocrats. "Capitalism" and "Socialism" become curse-words for each, representing the absurd extremes of both philosophies, when their true definitions and practicioners are really the sane versions.
But it is important to note that, while this philosophical axis aligns fairly closely with the popularized "left vs right" axis when dealing with economic policy issues, it is far less in alignment on social policy issues. The popularized Rightist policy of total gun rights and zero gun control aligns with the Individualist philosophy and is counter-balanced by the popularized Leftist policy of total gun control and total gun bans aligning with the Collectivist philosophy. HOWEVER, the popularized Rightist policy of total abortion control and total banning of abortion is a COLLECTIVIST philosophy, while the counter-balancing popularized Leftist policy of total abortion rights and zero abortion control is an INDIVIDUALIST philosophy.
There are several reasons for why the popularized left-vs-right axis aligns along the Collectivism-vs-Individualism axis so well on economic issues but less so on social issues. One of the biggest is the Cold War. The Cold War was defined by the conflicting economic policies of Insane Collectivism and Sane and Insane Individualism (the Insane-Collectivist philosophy of Communism directly threatens both Sane-Individualism Capitalists and Insane-Individualism Feudal Aristocrats) and Sane Collectivism and Insane Individualism (the Sane Collectivist philosophy of Socialism directly threatens Insane-Individualism Feudal Aristocrats), and its impact on our culture and society was massive. So many of our memes and concepts and thought processes were built upon that foundation that even today, twenty years after the end of the Cold War, we still naturally frame our thought processes along those lines.
The other biggest reason is that economic policies and issues are so broadly encompassing that they impact everything, while social policies and issues are largely very specific and tend to be segregated from each other, both in their effect on each other and the origins of the issues and perspectives on those issues.
Before I can get into the WHYS of the push behind this division, I must address the other two major divides.
Look-Forward Optimism vs Look-Back Nostalgia
The next major divide is between Look-Forward Optimism and Look-Back Nostalgia. These are more mindsets than philosophies, the way one perceives the world, but they also include who one responds to those perceptions, and the general behaviors associated with the two perception regimes.
A Look-Forward Optimist looks forward to see a golden age in a bright and glorious future. These are the dreamers, the visionaries, the adventurers and the doers who go out to find that better tomorrow and make it real. Some of them are naive, they have a fancy image in their head with no idea of the kind of work and effort it would take to accomplish it, or they can't properly see the potential negative consequences of the new ideas they are fixated on, but most of them have at least some functional sense of risk-appraisal and are at least partly aware of the work needed to make their dreams reality. They are an energetic and chaotic lot, often reckless and ever curious. That can get them into a lot of trouble, and some of the worst acts of evil in our history have been committed by twisted members of this group, but almost all of the greatest achievements and advancements of our civilization have been accomplished by or driven by the sane members of this group.
A Look-Back Nostalgic looks back to see a lost golden age in a glorious past. Often romanticizing or glorifying eras that weren't really as great as they seemed, they are wary of the future and change, and wish to undo changes and revert to their past era of idolized glory. Look-Back Nostalgics can serve as a needed damper to the rambunctious Look-Forward Optimists, keeping them from running off pell-mell with new ideas and toys without properly considering and protecting against the potential dangers, but they have also been responsible for some of the worst atrocities in history (both directly, and indirectly by delaying or debilitating much-needed changes and advancements).
While this philosophical axis currently tends to align with the popularized left-vs-right axis with the Look-Forward Optimists tending to fall into the general category of the so-called 'left' and the Look-Back Nostalgics tending to fall into the general category of the so-called 'right', this alignment is hardly perfect, and varies between different times and cultures.
The conflict between these two mindsets comes when the Look-Forwards push too hard or fast for change, and/or when the Look-Backs dig their heels in and rail against needed change too hard. There is always some conflict, which isn't a bad thing, but too much conflict causes problems. The current conflict is a mix of both, but originates with the Look-Backs, who are pining away for lost golden ages of yore after three centuries of remarkably rapid, and rapidly accelerating advancement. The Look-Backs' role in a healthy society is to make sure we haven't made things worse than they are, but that role breaks down when they fail to truthfully appraise the realities of the past and recognize the accomplishments made, and when they fail or refuse to recognize the challenges and problems faced that require us to change.
The cause of this break-down has to do with the last major divide.
Rationalism and Reality-Based Thinking vs Fantasy-Based Thinking and Embracing Cognitive Dissonance
This last philosophical divide is only one of the three listed here that truly is mutually exclusive and that has a definite 'wrong' side. It is also the one that has the least correlation to the popularized left-vs-right divide, though there IS still a definite correlation.
Like the Look-Forward-vs-Look-Back axis, this axis is more of a mindset, how people perceive the world, their mode and method of thinking and making decisions.
Reality-based thinking is basing one's knowledge, thought-processes and decision-making on what is real. Reality is defined as what happens and what exists regardless of what you believe will happen or will exist. If you walk off a cliff believing you will fly, and you fall, the cliff is real, gravity is real, and whatever force you believe will make you fly is not. Rationalism is the method of thinking that allows one to determine what is really real, to determine the truth, and how to process those truths to make correct and effective decisions, and how to compensate for the natural flaws in human nature and how the human mind works and processes information.
Fantasy-based thinking and embracing cognitive dissonance is basing one's knowledge, thought-processes and decision-making on what is believed to be real, what one feels to be real, and what one wants to be real, regardless of what is really real, and the embracing of our cognitive dissonances, the flaws in human nature and the human mind that allow us to hold conflicting beliefs simultaneously, or to believe in something that isn't real despite irrefutable evidence to the contrary, or to make decisions that are obviously wrong to outside observers despite knowledge and experience that should let us know better.
This divide between rationalist, reality-based thinkers and fantasy-based thinkers who embrace their cognitive dissonances is spread across all political spectrums, philosophies and ideologies because we all suffer from the same flaws of human nature and the same flaws in the hardware and underlying software of how our brains work. Not everyone even knows to watch for and correct these flaws in how we think, let alone how to effectively go about doing that, or tries to, and even the best rationalists are not perfect.
This divide does still tend to fall along the popularized left-vs-right axis, however. It's not a perfect correlation by far, but there currently IS a definite correlation between rational, reality-based thinkers and political views and positions that fall into the general category of the so-called 'left', and cognitively dissonant fantasy-based thinkers and political views and positions that fall into the general category of the so-called 'right.'
This correlation is most clearly seen in the political positions and views of our nation's scientists. Scientists are, by the nature of their careers and education, rational, reality-based thinkers. The Scientific Method is an intrinsic part of the Methods of Rationality, and is designed to determine truth based on verifiable, testable, repeatable facts of reality. Scientists are trained to use the Scientific Method and to think scientifically, and the scientific community is set up so that they provide reciprocal accountability to each other. The peer-review process, a critical step in the Scientific Method, is a process where fellow scientists and experts deliberately rip a work apart, looking for any and all flaws in the data, testing methods, conclusions, math, any errors.
The political affiliation of scientists is striking. Only 6% of U.S. scientists are Republican, while 55% claim to be Democrats, 32% Independent, and 7% uncommitted. When the best-educated, most intelligent, most knowledgeable, most rational, most reality-based thinkers in the country correlate to a general political ideology, and near-unanimously correlate AWAY FROM another general political ideology, that says something VERY striking, and very much worth noting.
This does not mean that all so-called Leftists are rational, reality-based thinkers; there are plenty of irrational people on the so-called 'left.' A previous roommate of mine is prime example; a self-avowed feminazi, she absolutely refused to even allow the consideration that her position on abortion might be wrong in even the slightest way, or that any of the arguments of 'the other side' might have any kind of validity at all, and got very irate at even the suggestion of the idea that there might be even a smidgen of validity to any anti-abortion argument. Like most ideologues, she was addicted to self-righteousness, and clung to her positions and views with the blind, unthinking, militant faith of a religious fanatic (for the record and full disclosure, I myself do not have a set position on abortion, and see validity in points raised by both sides of the issue).
There are usually plenty of irrational ideologues addicted to self-righteousness in any world view or belief system, and it would not be correct to say that the so-called 'left' has fewer irrational people than the so-called 'right', but there is a very strong correlation between rational, reality-based thinking and moderate or 'left'-leaning political views. It would be more accurate to say that the so-called left has more rational thinkers than the so-called right, in the U.S. today (this varies by country/culture and time period, but the focus of this post is the current state of U.S. politics).
There are a few reasons for this, and a lot of it has to do with the Cold War. Because of the Cold War, and the fact that the U.S. fell in on the side of Sane-Individualist Capitalism against Insane-Collectivist Communism, Insane-Collectivist ideologues were largely purged from our political sphere, leaving only a handful of crazies, that everyone recognizes as crazy, promoting Insane-Collectivist ideologies, while any ideology or group that opposes Communism has received greater respect and acceptance for that alone.
Another major reason why far-left extremists are so rare and non-influential in U.S. politics is because of what is known as the Miracle of '47, and the two different paths our left-leaning and right-leaning political parties took. Back in the mid-40s, the Democratic party was seeing increasing radicalization from their far-left base, the actual communists and Soviet sympathizers. In the face of this radicalization, and the new threat from the Soviet Union, Democratic party leaders met in conference in '47, and a decision was reached to cut ties with their far-left branches. It was thought that it would severely weaken and perhaps even destroy the Democratic party, cutting ties with their hard-core base, but the concerns over the increasing radicalization, and also the concerns about the increasing number of Soviet sympathizers, led the Democratic leaders to decide that the party, and the nation would be better off with a weakened Democratic party over an increasingly radicalized Democratic party full of Soviet sympathizers. By 1949, this process of cutting ties with the true Communists and Soviet sympathizers had largely been completed and the Democratic party had shifted towards a more moderate, centrist position. Contrary to expectations, this actually strengthened the Democratic party, gaining them more of the critical moderate voters, at the expense of the increasingly marginalized radical left-wing voters.
When faced with a similar rise of radicalization in their own party in the late 70s and especially the 80s and early 90s, the Republican party took the opposite approach, and began catering to their far-right base, and have been doing so more and more over the last 30+ years. This has led to an increase in the strength and influence of radical and militant voices on the so-called right, in no small part because the more moderate, rational voices have been driven out, labeled "Republican In Name Only" or "Not conservative enough" (hence why it is more accurate to say that the so-called left, which has embraced science and scientists for decades, has more rational, reality-based thinkers than the so-called right, which has been attacking science and scientists for decades, than it is to say that the so-called left has fewer irrational thinkers than the so-called right).
Now, this whole political divide is not something that has developed on its own, there have been definite forces driving it, and there are definite forces driving it very hard today. There is no global conspiracy theory of the Illuminati or any such silly supposed all-knowing, all-connected, all-powerful secret society, but there is probably a loose 'conspiracy of mutual interest' going on. Not an inter-connected, highly-organzed, precisely-controlled global conspiracy with a Grand Plan and Evil Agenda, but the loosely-affiliated or independently parallel action of powerful players who have shared mutual interests. Oil, coal and gas tycoons, for example, all have a combined interest worth trillions of dollars vested in the denial of their products causing any serious or lasting harm, middle-eastern oil princes have vested interests in maintaining western dependence on oil, while also having vested interests in seeing the spread of and success of western (and eastern) culture (which both run counter to their own) collapse. Investment bankers and stock-market tycoons who have been benefiting greatly from loopholes and gaps in regulations that let them game the system have a vested interest in blocking any attempts to close those gaps. Politicians who have taken advantage of the power and safe-vote guarantees of political polarization have a vested interest in maintaining that polarization and destroying any attempt to bridge the divide and cooperate. Religious and ideological fanatics have a vested interest in pushing that divide because they see 'the other side' as The Evil Enemy and view any compromise with The Evil Enemy as collaboration with The Evil Enemy. And the Insane-Individualist Feudal Aristocrats (who have been the main force that has dominated and oppressed societies across 14,000 years of human civilization) have a vested interest in undoing the progress of the last 300 years that has bootstrapped the unwashed peasant masses up from abject poverty to the vibrant and (most importantly) empowered working- and middle-class, to maintain their positions as society's ruling elites, and reclaim their oligarchic positions that have been undermined or removed completely by the Renaissance and especially the Enlightenment and Progressive movements in the last 300 years.
Insane-Collectivism is NOT the main driving force behind the gaping divide and bitter feud in American politics and Western politics in general. Insane-Collectivism is a new force that has only recently come into existence in the last 200 years, and that never truly came into full fruition. Insane-Individualism has too much power and influence, built up over 14,000 years of human civilization, America's taking up of the Individualist side of the Cold War all but exterminated Insane-Collectivism from our political sphere, and most implementations of Insane-Collectivism have ultimately been coopted by Insane-Individualists. The Soviet Union is a prime example the latter. They never truly fully implemented Insane-Collectivism in the Soviet Union. Their Communist system was thoroughly corrupt, and was mostly a cover for Insane-Individualists to claim power. The Feudal Aristocrat Czars were overthrown by revolutionaries who lacked the levels of education, and personal ethical and moral fortitude of our own revolutionaries. After over-throwing the Czars and declaring Communism, they succumbed to temptations of the concentrated power that is inherent to government (especially Communistic government, which combines the concentrated power of government with the concentrated power of total economic monopoly), without having a strong system of checks and balances in place to curb and regulate government power and prevent and minimize abuses of government power.
(It really can't be stressed enough how fortunate we were to have the revolutionary leaders we did; they weren't perfect and didn't have the solutions to every problem, but they did a remarkable job of setting up a system to balance against corruption and abuse of power, and resisted incredible temptations themselves. George Washington is a prime example. Not only did he personally insist on resigning his military commission before being sworn in as President, he also personally refused an offer to use his military command to seize power and establish himself as king not once, but three separate times!)
There has also never been a full implementation of ultimate Collectivism. We have never had a Borg Collective in the history of our civilization. It's just not something that has been possible to do. It might become possible in the future with new technology that is just coming over the horizon, and the ultimate implementation of Insane Collectivism is a future threat we will have to diligently guard against, but it is not a real threat we have ever faced in the past, and it is not a threat we face today.
The great threat to freedom, equality, justice, independence and open markets across 14,000 years of human civilization has been Insane-Individualism and the oligarchies of Feudal Aristocrats. They have been ingrained in our civilization for 14,000 years, they have a power base built up over 14,000 years, and they are a threat to, and threatened by freedom, equality, justice, independence and free, open markets just as much as, and perhaps even more than Insane-Collectivism.