These two books are among the first fruits of this new industry. Their authors are influential participants in Washington’s hyperactive marketplace of ideas, and they each offer distinctive perspectives. Francis Fukuyama is perhaps the most widely quoted foreign-policy pundit on the planet. In the early 1990s he declared that the end of the Cold War meant the triumph of market democracy and the end of history. These were key ideas in the neo-conservative assault on American foreign policy. Fukuyama counted himself among the neo-cons, and at first supported the Bush Doctrine. His book formally repudiates this affiliation, and offers in its place a new policy precept which he calls ‘realistic Wilsonianism’. Anatol Lieven and John Hulsman approach the problem from both sides. They present themselves as a coalition of opposites, uniting to forge a new consensus about America’s global role. Lieven is from a centrist think-tank; Hulsman worked for the distinctly right-wing Heritage Foundation. They find fault not just with the neo-cons of the right but also with the liberal hawks of the American left who share some of the neo-cons’ views. They offer a philosophy they call ‘Ethical Realism’ as an alternative to both.

Both books have a lot to offer. They describe in sensible terms what is wrong with the Bush Doctrine, and they explore the options and alternatives by drawing impressively on the history and traditions that shape the ways Americans think about foreign policy. They provide, therefore, a measure of reassurance that America can recognise the mistakes of recent years and find a better approach for the future. But only a measure of reassurance, because what shines through these intelligent, learned and well-intentioned works is how deeply embedded in American political consciousness are the ideas that gave rise to neo-conservatism, and how hard it is for America to escape them.

Ethical Realism opens with a gloomy view of America’s situation. The risks are higher, its authors say, than in the Cold War. Like most people in recent years, including the neo-cons, they emphasise at first the threat from global jihadist terrorism. To meet this threat, America needs the kind of fresh thinking and visionary policies with which Presidents Truman and Eisenhower laid the foundations for America’s eventual victory in the Cold War. They draw inspiration from three of the most prominent policy intellectuals of the Truman–Eisenhower era: Reinhold Niebuhr, Hans Morgenthau and George Kennan. From their ideas, Lieven and Hulsman construct their philosophy. Ethical realism, they say, is characterised by ‘prudence, patriotism, responsibility, study, humility, and “a decent respect” of the views and interests of other nations’.

This is all a little pompous, perhaps, but the principles themselves are sound and provide an excellent frame for the authors’ mordant examination of the follies of recent American policy. They scrutinise and criticise all the key neo-con ideas: preventive war; the promotion of democracy; the universality of American values and institutions; the glorification of armed force; the exaggeration of American power, both ‘hard’ and ‘soft’; and the confusion of the kind of violent jihadism that gives rise to terrorism with wider processes of social and political development in the Islamic world.

They propose instead that American foreign policy should abandon plans to reshape the world, and focus on more modest and cooperative efforts to support what they call ‘the Great Capitalist Peace’. They use this ungainly phrase to refer to the international system of cooperation among major powers which has done so much to underpin global prosperity in recent years. They subtly shift attention from new non-state threats like terrorism to the traditional concerns of international power politics. They say America’s key priority should be to stabilise relations between states, by extending their Great Capitalist Peace to areas which are as yet outside its orbit, like the Middle East, and to reinforce it in places like Asia, where it is under challenge from rising powers.

This is an inherently collaborative project, involving close cooperation with other states. That means America has to learn to respect the interests and institutions of other states, and stop trying to throw its weight around. It needs to foster the emergence of regional ‘concerts of power’ in the Middle East, Asia and elsewhere, modelled on the European Concert of the nineteenth century.

It is hard to argue with any of these ideas. This is the kind of American foreign policy that non-Americans have always wanted to see, in which America works in and with the international community, not against it, and adopts old-world traditions of careful diplomacy, accommodation and compromise. Indeed, it seems too good to be true, and so it proves. Because the big question is, will Americans buy it? Lieven and Hulsman are really proposing something very close to the traditional school of realist foreign policy espoused by Henry Kissinger. Realism has always sat uneasily with Americans, because it runs against the deeply held conviction that their country is not like other countries; it stands apart, unique and special.

This conviction is still strong, even after Iraq. As Owen Harries has recently said at the Lowy Institute, what is called American exceptionalism has survived the collapse of the Bush Doctrine and remains central to American’s approach to the rest of the world. It poses the critical challenge to anyone wanting to build a durable replacement for the Bush Doctrine. How does Washington conduct a foreign policy that is at once true to America’s values and its sense of a unique mission to civilise the world, and at the same time accommodates the brute facts that Bush and the neo-cons ignored: that America, for all its power, is not strong enough to get its own way everywhere on everything; that its values do not attract universal respect and its conduct often provokes opposition; that America, in its own interests, has no option but to work with others, and must adjust its aims and agendas to accommodate others’ interests? This has been the big question in American foreign policy for a century now.

Lieven’s and Hulsman’s new grand plan only works if they can reconcile American exceptionalism with their realistic understanding of America’s need to work with others. They hope to do this by marketing their philosophy as ethical realism, and much of their book is taken up trying to persuade their American readers that being ethical makes it different from the old kind of realism by being true to America’s values and its destiny as the ‘last best hope of mankind’. It is worth reading the book to see how they go about it, but in the end I fear they do not succeed. The give-away is that they do not seem even to have convinced themselves. When they come to illustrate the practical applications of their philosophy, their own exceptionalist instincts come through.

They have much that is sensible and even courageous to say about such sensitive and critical issues as America’s relations with China, but in other places Lieven and Hulsman slip back into forms of words and modes of thought that suggest that they too think that America does after all know best, and does indeed have the power to get its way. They say Iran should be ‘allowed’ to enrich uranium but not to build a bomb, ignoring the fact that America does not seem to have any practical way to stop it. They say that Russia should be forced to crack down on Iran, as if America had ready means to control Russian policy on this or anything else.

In the wider Middle East, they say America should ‘veil’ its power behind regional states and regional agreements, as if the regional concerts they propose are not really about consultation and compromise, but merely a façade to hide the old American unilateralism. So in the end it is not clear that Lieven and Hulsman have made much progress at all towards resolving the contradictions inherent in their effort to reconcile American exceptionalism and realism. It is the old hubristic neo-con wolf in some not very realistic sheep’s clothing.

ukuyama’s book has a cooler tone; more detached, less pompous, more analytic. It contains some really interesting material. There is a fascinating and fair-minded account of the intellectual origins of neo-conservatism in the work of Leo Strauss, Allan Bloom and Albert Wohlstetter, and of its social origins among disillusioned leftist students at the working-class City College of New York in the late 1930s. Fukuyama describes how neo-conservatism changed with the end of the Cold War, and how a new younger generation of neo-cons, grown overconfident after what they saw as the vindication of their ideas with America’s ‘victory’ in the Cold War, hijacked the movement to create the intellectual foundations of the Bush Doctrine.

Fukuyama identifies the same faults in the Bush Doctrine as Lieven and Hulsman, and he too proposes a more realist policy of consultation and compromise as an alternative to its hubristic unilateralism. But he approaches the development of his alternative rather differently. For a start, Fukuyama is more relaxed about America’s security outlook. He says Bush and the neo-cons have exaggerated the terrorist threat, and offers, in a few superbly crisp pages, a clear and sceptical corrective. Moreover, he does not seem at all worried about the future of America’s relations with other major powers such as China, taking for granted that there has been ‘a shift in the focus of action away from nation-states towards non-state actors and forces’. As a result he sees ‘the emergence of a band of weak and failed states that are the source of most global problems’. Strengthening these weak and failed states is therefore America’s main foreign-policy priority, and true to his own neo-con roots, Fukuyama argues that this requires the creation of new ‘regimes’ in these states.

He parts company from his old neo-con colleagues, however, when he explains what this involves, and how it is to be done. In doing so, he attempts to rescue his own ideas from the wreck of neo-conservatism. Fukuyama distances himself from the neo-con interpretation of his ‘end of history’ thesis as predicting the swift, sure or easy rise of market democracy around the world. He states that he always believed that because political institutions are so intimately connected to the life of their societies, regime change is a slow and complex process that cannot be done from outside. He writes at some length and with great insight into the problems that the West has in trying to help weak and failing states rebuild themselves – a topic highly pertinent to Australia’s state-building enterprises in its own backyard. In particular, Fukuyama urges that state-building cannot be done with armed force, nor can it be done by America acting unilaterally. America, Fukuyama says, lacks the legitimacy to foster regime change in other countries unilaterally.

Instead, America must learn to work through international institutions, and Fukuyama says America’s most urgent task is the construction of a network of multilateral institutions for this purpose. Here, in a different guise, he runs into the same problem as Lieven and Hulsman. Americans have mostly been deeply ambivalent about multilateral institutions of which they cannot unambiguously take charge, precisely because the negotiation and compromise inherent in any genuine collaborative enterprise jars with their sense of exceptionalism. Like Lieven and Hulsman, Fukuyama needs to find a way to reconcile America’s need to work with other states with the deep-seated instincts of American foreign policy. Fukuyama invokes the name of Woodrow Wilson – architect of the League of Nations, but also one of the high priests of American exceptionalism – to convince his readers that international cooperation can be reconciled with America’s unique standing as a beacon to other nations. But he ends up being even less successful than Lieven and Hulsman in doing this. Fukuyama’s curious decision to invoke Wilson’s memory itself serves as a warning. Wilson, of course, failed to persuade Americans to join the League he had created.

Both books, then, are full of interest and good sense, but they leave one rather gloomy. Both fail in their attempts to resolve the tension between the real limits to American power and influence, so brutally shown in Iraq, and America’s enduring sense of itself as a country apart from, and above, all others. Their failure shows how hard it is for Americans to conceive of their country as a normal member of an international society of states. And that spells trouble ahead, for how can America be anything else? In the end, for all its power, America is not strong enough to rule the world. It has to work with others, and compromise with them; and this need will only grow as new centres of power emerge, especially in Asia. Can America abandon its exceptionalist instincts and accept this? For example, will any American leader stand up and explain to American voters that as China grows they will need to accept China as an equal? Can stable relations with China develop if they do not?

This is a critical question for us here in Australia, because United States–China relations may well shape our strategic future more than anything else. To get it right, America will have to find a way to resolve this challenge, and that means it will have to change the way it thinks about itself and the world in fundamental ways. Since it became a major power more than a century ago, America has never related to other great powers as a cooperative equal. It has always seen others either as followers or as adversaries. Can it learn a new approach now? Not without much harder thinking than we find in either of these books, impressive and worthwhile though they are.