security research, software archaeology, geek of all trades
243 stories

Finding Hope in a Loveless Place


By January 2016 I had made peace with the idea of a President Donald Trump.


It was a strange statement for a black woman, a sociologist, a professor. I would go on to consult for Bernie Sanders’ campaign in the primary. I had written an essay for a mainstream anthology about Hillary Rodham Clinton’s brand of feminism. It wasn’t a secret that I was unsettled by how the Democratic party wanted to approach this election.


Still, it was a strange notion to confess to in public, which is exactly how I did it:




Professionally smart people were still very dismissive of any idea that Trump could win the primary, much less the presidency.


Then Trump won the nomination.


Professionally smart people then told us about how polls work and about political theory. The United States of America couldn’t elect Trump. We weren’t that nation anymore.


I have been black a long time and I have been southern for about as long. I’m talking at least one lifetime if you don’t count the culture trapped in my bones, inherited from my people.


I am trained a social scientist. I know how polls work, much to the chagrin of the professionally smart men who sneered at me when I said this nation could absolutely elect Donald Trump.


To be professionally smart is to concede always to rational science, to polls and confidence intervals. These colleagues, the professionally smart, seemed dismayed that the black woman they’d been brave enough to think smart could believe a President Trump was possible.


They were dismayed but not surprised.  Women and black people always have a potential blind spot where race and gender are concerned. It is why we’re so emotional and irrational. We just cannot see past our unscientific claims of racism and sexism to be truly professionally smart. Our models, in the parlance of the professionally smart, are always just a bit skewed.


It’s a shame, too. Because the professionally smart really want evidence that their faith in affirmative action for smart minorities is well-placed. It is a shame with a black woman has as much potential as me and still can’t see past that racial blind spot.


My blind spot was, of course, perfect clarity about how whiteness and racism work.


It is a difficult thing to measure in polls. That’s why there is still great value in systematic collection and analysis of how people experience the world and not just how they tell you they experience the world.


People told pollsters that they were undecided or that they weren’t sure if they would vote. People told themselves that they were rational and not victims to blind spots or identity politics. People told themselves and told us that this is not the nation that could elect a liar and a cheater and a bully and an ingrate and a white nationalist President of the United States of America.


But people went to Donald Trump rallies.


I know because I went to one, too.


I didn’t just ask people why they were there. I paid attention to how they were there. Did they bring their children? How did they comport themselves? How were they dressed? What cars filled the parking lots? What were the side conversations in stairways and lines for the bathroom?


At the Donald Trump rally I heard white women swoon at the thought of catching his eye from the stage. Women in their fifties giggled like school girls over how sexy he is. Young but infirm white men and women on crutches and in hoverrounds and walkers with the little wheels on the back legs screamed about how Obamacare was ruining America.


And black men sat silently in corners, nodding almost imperceptibly when he talked about trade. South Asian and east Asian groups for Trump littered the hall with signs and t-shirts showing their support. Young people cheered when Trump said he would arrest Hillary and cage in the inner cities, wearing jerseys fresh from or to some sporting practice.


This was America and I knew it was because for me it always has been.


Johnathan Chait once chided Ta-Nehisi Coates for his hopelessness. If there is a more persistent demand of the marginalized and oppressed than that they perform hope for their benefactors, it is difficult to find it. We have, of course, a nomenclature problem. When white allies want us to be hopeful what they really mean is that they require absolution in exchange for their sympathies. And, when black people say that they are plenty hopeful we tend to mean that our hope is tempered by a deep awareness of how thin is the veneer of white civility. Our grudging acceptance that progress and diversity are fragile bits of spun glass looks like hopelessness because it doesn’t absolve. But, it is the most enduring kind of hope and it is the hope that President-elect Donald Trump will require of us all if we’re to organize and resist.

Professionally smart people who claim to know our collective heart really like straight lines. First, there was the enlightenment and then some unfortunate business about colonialism and black ocean bottoms. Then there was freedom and the Great Society and smartphones and the browning of America.

The straight line requires a lot of erasers.

Sociologist Karen Fields and historian Barbara Fields have this wonderful analytical concept called racecraft. I like it a lot for many reasons. Chief among them is the clear articulation about how rational hopelessness is and how radically necessary it is to any kind of progress for a society so embedded in racial hierarchies as is ours.

Racecraft is the great story about what race is and what it can never be for race to persist as a powerful social category. Racecraft is the debate about whether Trump voters acted out of partisanship or economic anxiety, out of sexism or racism because plain old racism is simply too blunt a tool to explain such complex social behavior. Mind you, partisanship literally means just voting like lots of other people vote out of loyalty. Racism, in contrast, is a cohesive analytical framework that accounts for the complex ways that identities and ideologies shift over time. One of those concepts is as complex as the phenomenon in question but only one is dismissed outright as foolish. When racecraft is the prevailing responsibility of professional smart people and erasers and line drawers, these kind of inconsistencies don’t matter.

Racecraft necessitates that facts and truth, as Mark Zuckerberg intoned this week when Facebook was indicted for facilitating the spread of fake news stories, are always fundamentally unknowable. Philosophically, of course, there is a debate about whether there is any such thing as “truth” but politically there is no doubt that there is an agreed upon idea of what constitutes a fact. A social fact, even, as sociologists often call it. How ironic for the great narrative emerging from this election to be so fastidiously addicted to “fake news” that may or may not have swayed the election given how racecraft has shaped the political fabric of this nation, eating truth whenever convenient to make sure that fabric only ever secured white freedoms.

Racecraft tells us that any truth, any fact can be dispensed with when it doesn’t support the straight line. Racecraft has to allow for science that refutes the existence of races while also proving that there is such thing as races. Racecraft is a nation’s split personality that must have a measurement device for popular opinions while discounting one of the single greatest predictors of popular opinion. Racecraft is how the only folks who see things clearly can be the ones accused of having a blind spot.

That is why professionally smart people require all members to the club have irrational, exuberant hope. But, the black members have to have the most irrational and exuberant and performative hope. We cannot just scream “yes we can!” We have to scream it even when it is clear that we cannot and no, we will not and no, we are not. We have to pay this tax to the professionally smart because our blind spot is so discomfiting. We come with the baggage of lived experience of racism.

You hear it already, the calls for liberals and democrats to get back to hope as quickly and efficiently as possible. The call is loudest and most pointed when aimed at the black voters and the women voters, god help us, the black women voters. They cannot be bothered right now with our casual hopelessness that has always already lived with the dystopia promised by a President Trump. Everything from reversing Roe v. Wade and police surveillance to mob rule in public discourse and delegitimized claims to citizenship have been true for us for a long time.

The America many professionally smart people woke up to last week is the America many of us have already lived in for at least as long as this memory shut up in my bones.

The irony is that despite our hopelessness, black voters worked overtime to defeat Trump. We did this despite having more barriers to voting than almost any other part of the electorate: gerrymandering, voter ID laws, time constraints, and voter intimidation. Just like we have always showed up. We have showed up to free ourselves, to free the immigrant others who came after us, to perfect this union against its will, and to make democracy great one day.

My hopelessness isn’t nihilism just like my blind spot has always seen clearly the limits of American progress.

My hopelessness is faith in things yet seen and works yet done. Hopelessness is necessary for the hard work of resisting tyranny and fascism. It is the precondition for sustained social movements because history isn’t a straight line. It is a spinning top that eventually moves forward but also always goes round and round as it does. Those erasers applied post-mortem confuse us to this, blind us to the defeats that will come and ill prepare us for the reality that most of what we believe in will not come to pass in our lifetimes. A transactional hope is anathema to social progress.

I knew this America could elect a President Trump. It is precisely because I always knew it, bone deep, that I worked so hard to stop it.

It’s why the work never stopped.

For some of us it never has.

Read the whole story
1 day ago
Mountain View, CA
1 day ago
Washington, DC
Share this story

Hey, Looks Like It’s Time Once Again For Me to Talk About Writing On Politics


Because of the election and all, I’ve gotten a few people griping to me about the fact I write about politics here and in other places. It’s been a while since I talked extensively about me writing about politics, and also, about the more general topic of entertainers and creative people who talk about politics, and the people who tell them to shut up about it. So let’s talk about these things, shall we.

For ease of discussion, I’ve broken this up into ten points. The first five are about me specifically, and are short. The second five are more general, and rather longer. Ready? Let’s get to it.

I. The Short Points About Me Writing On Politics

1. If you tell me you’re tired of me talking about politics, or tell me to shut up about them, I’ll tell you to kiss my ass. I’ll write about what I want, when I want, where I want, which in this case happens to be about politics, now, here.

2. If you don’t want to read me opining on politics, you are presumably a grown human being with free will and the ability not to read things. Skip over the piece or stop reading the site entirely.

3. If you complain to me about my expressing political opinions in areas under my own control that you are not actually being forced to read, there’s a very good chance I’m going to be rude to you. This is actually covered in the site disclaimer, which has been up for years, on every page of the blog.

4. I don’t give a shit if you become unhappy with me for being rude to you.

5. Likewise, I don’t care if your dislike of my writing about politics and/or your being upset that I was rude to you when you complained about it means you no longer choose to read my books. Stop reading my books, then.

II. The Longer Points About Creators and Entertainers, Including Me, Opining On Politics

6. For the occasional jackass who opines that entertainers like myself should stick to entertaining and not write about politics or anything else that might possibly offend someone, a) fuck you, b) you’re wrong, c) independent of either of those points, long before I was an author I got paid to write about politics, and still do from time to time (as recently as last month, in fact, in one of the largest newspapers in the nation). So, yeah, actually, writing about politics is a thing I do professionally, thanks so much for asking.

Now, here’s a hot take for you: “Entertainers” are fully-dimensional human beings who don’t exist solely to entertain you, writers in particular write professionally and/or critically about many things over the course of their career, and you suggesting that people not express themselves about politics (or anything else) because they are an “entertainer” makes you the asshole in that scenario, not them. So maybe don’t be shocked when they tell you to sod off.

7. Likewise, this blog existed before I was a published author, and before I was a published novelist. I’ve been writing about politics here literally since the very first week it was up (that week I also wrote about baseball, journalism, my pets, tech stuff and sending out invoices. The site lived up to its name even then). The blog isn’t here to sell my books, although it’s done that incidentally, which makes me happy. It’s here for me to write about whatever I want to write about, when I feel like writing about it, and has been since 1998. Sometimes I will write about politics! And sometimes I won’t — there are entire years (see: 2014) where I pretty much didn’t, because I didn’t want to.

Which again is the point of the blog: It’s about me, writing about what interests me, when it interests me. It’s not about you, or what you want me to write about, or to not write about. Likewise, my Twitter feed, or anything else I control that I put my words out on. If you’re confused about the vector of impetus and control regarding these outlets, for any reason, you’re likely to have a bad time. Especially if you complain to me about it. Other entertainers and creative people may feel similarly about their own spaces, which is a thing you should consider as well.

8. With that said, if my politics make you itchy, then actually, you should probably consider skipping out on the next four years here. I can’t imagine I’m not going to write about politics on a regular basis. I mean, come on: whatever one thinks about Trump, that asshole isn’t boring. Nor are American politics or social issues going to be anything but a bumpy ride the next few years. I’m not going to say it’s going to be fun. But it’s not going to be something I’m likely to take a pass on as a writing topic. If you want a spoiler alert on the matter, this is it. Don’t expect me not to go off on politics, folks. And don’t expect other creative/entertaining folks to be quiet about politics either, the next four years. That’s just not gonna happen.

9. Why, yes, I’m cranky on the subject of people trying to tell me — or other writers, creators and entertainers — to stop expressing opinions on politics and other social issues. People have been trying to do this for my entire professional writing life, which is 26 years now, and much of which has consisted of me being paid to have a goddamned opinion on things. I imagine it will continue, as people with fragile worldviews and/or monstrous senses of entitlement and/or a wildly misplaced sense of my desire for their input and/or simple, virulent passive-aggressiveness decide they need to tell me what and how and when to write, or not write.

And, folks. One: Are you my spouse or my editor? No? Then feel free to fuck right off. Two: You may have noticed I’m doing pretty well for myself with this whole writing thing. One secret to that is not listening to various randos telling me what to do, or what not to do, with my writing career. Three: Do you understand how boring and exasperating it is for me — or any writer, creator or entertainer — to have to deal with various randos telling us what to do or not to do? It’s really goddamn boring and exasperating! And maybe other folks who have to deal with this bullshit choose to be patient or quiet about this because they’re earlier along in their career, or they’re still under the impression that their career can be hurt by some rando telling them to shut up or else, or because they’re nicer than me, or because they’re not, like me, a well-off straight white dude so they actually have to worry about their randos being terrifying stalkery bigot assholes, especially now, when actual fucking Nazis are cracking off salutes like it’s 1933. All of those are fine reasons for them to be quiet about this shit.

But I am not them, so I’m pretty comfortable saying the following: Piss off, rando yutz, you’re boring me. After 26 years, you’re not going to find a way to tell me to shut up that I haven’t heard before and haven’t already offered a middle finger to. And after 26 years, I’ve run out of fucks to give on the matter. Tell me to stop writing about politics? Fuck you. Suggest that I shouldn’t write about politics? Fuck you again. Whine to me that you’re tired about me writing about politics? Fuck you a third time. There’s the door. Go.

10. Seriously, people, what do you think you’re doing when you tell a writer (or musician, or actor, or whomever) that they shouldn’t talk about politics, or social issues, or whatever? Do you believe they will genuinely think, “My god, this random person I don’t know has entirely changed my mind about expressing an opinion in public! I am so grateful”? I can’t speak for all creators everywhere, but anecdotally speaking, I can tell you that most of the creators I know do not think oh wow, this random person is so right. They think, what an asshole.

And maybe you are an asshole! Certainly there are any number of people who send me notes along the line lol shut up dude no one cares what you think, which aren’t meant to persuade, but just to try to insult or belittle me, and, well. That’s adorable. But if you didn’t intend to be an asshole, maybe consider a different strategy.

For example, a couple of years ago there was a Scottish entertainer I admired, so I followed their Twitter account, which turned out to be nothing but blathering about Scottish independence. Did I tell this entertainer to oh my God will you just stop talking about Scottish independence I don’t even care? No, I just stopped following their Twitter account. Because you can do that! There are whole swaths of creators and entertainers whose work I admire who I don’t follow on social media, or read their blogs, or otherwise track their lives because I know they care about things I don’t, or that I disagree with. There are other swaths of entertainers who I do follow, but when they get a bug up their ass about something I don’t care about, I skip over those topics. A writer I admire has gone on for years about vaping. I couldn’t give a shit about vaping; I think it’s a dumb thing to invest any brain cycles on. But they disagree! Good for them. I skip over the vaping rants. It’s really just that simple.

It’s okay to disagree, sometimes vehemently, with people whose work you admire. It’s all right to think they spend too much time on things you don’t care about. It’s fine to think to yourself or to tell others ugh why can’t they just get over that dumb thing I don’t care about. But the minute you go out of your way to tell them to shut up, no matter how “politely” you put it, you’re the asshole. Yes you are. And some of the people you’ve told to shut up will treat you like the asshole you’ve become.

I will, in any event. Fair warning.

Read the whole story
7 days ago
Mountain View, CA
Share this story

On Feelings and Votes

1 Share

This is going to be a bit of a rant, because there’s a recurring theme in my recent social media that’s really bugging me, and I need to vent. I’m going to do it as a blog post rather than an early-morning tweetstorm, because tweets are more likely to be pulled out of context, and then I’m going to unfollow basically everybody that isn’t a weird Twitter bot or a band that I like, and try to avoid politics until the end of the year. Also, I’ll do some physics stuff.

This morning saw the umpteenth reshared tweetstorm (no link because it doesn’t matter who it was) berating people who write about how liberals ought to reach out to working-class whites– as I did a little while back— for caring too much about the “feelings” of white people. While there are undoubtedly some disingenuous op-eds being written for which that’s true, I think it misses an extremely important point about this whole thing. That is, it’s true that these pieces are concerned about the feelings of white people, but only as a means to an end. What really matters isn’t their feelings, but their votes.

And all the stuff being thrown out there as progressives work through the Kübler-Ross model need those votes. You think it’s ridiculous that Hillary Clinton won the popular vote by 2.5 million votes but still lost, so you want to get rid of the Electoral College? Great. To do that, you need to amend the Constitution, which requires control of Congress and/or a whole bunch of state legislatures, most of which are in Republican hands, because they get the votes of those working-class whites. You want to ditch the Electoral College, you need to change those votes.

Think those working-class whites have too much power because of gerrymandered districts that over-weight rural areas? You’re probably right, but if you want to fix it, you need to control the legislatures that make the districts, and those are mostly in Republican hands because they get the votes of those people in rural districts. You want to stop gerrymandering and protect voting rights, you need to change those votes.

There are a whole host of things wrong with our current system. Fixing any of them requires winning elections, particularly those off-year legislative elections where Democrats underperform even when they’re winning statewide and national elections. Winning some of those is going to require getting the people who vote in those elections to change their votes, and hopefully their minds.

And that is why pundits and those who play pundit in a half-assed way on their blogs are saying you should care about the feelings of those working-class whites: because they vote, and you need their votes. And you’re not going to get those votes by berating them and insulting them and disparaging their feelings. You get their votes by understanding where they’re coming from, offering them something they want, and treating them with respect.

And again, this does not mean you need to cater to their basest impulses. Fundamental principles of tolerance and equality are not negotiable, and can not be compromised. But you don’t have to pander to racism to move some votes– most of the policies in the Democratic platform are already clearly better for those people than the Republican alternatives. It’s just a matter of pitching them in a way that makes that clear.

As an attempt at a concrete example, look at issues of affirmative action and immigration. If you’re dealing with someone who’s concerned about immigrants or people of color “taking our jobs,” you’re not likely to bring them to your side by lecturing them about how they’re not really entitled to that job, they’re just the beneficiary of hundreds of years of racist policy, and so on. You might be right about the history, but that’s not terribly persuasive to somebody who’s worried about having a stable income and health insurance to support their family. But you don’t need to go full “build a wall,” either– something like “The real problem is that there ought to be enough good jobs for both you and them, and here’s what we’re going to do to make that happen” could work. (It has the disadvantage of needing a plan to create jobs for all, admittedly, but as the recent election shows, such a plan doesn’t even need to be all that plausible.) That steps around the implicit racism of the original concern in a way that preserves their feelings, gets their vote for better policy, and doesn’t compromise any fundamental principles.

(Yes, this is basically the Bernie Sanders strategy. I would’ve been all for Bernie’s economic program; I don’t think he would’ve been a viable candidate in the general election, though.)

Another common and maddening refrain the past few weeks has been “Why do we have to care about their feelings, when they’re hateful toward us?” The answer is, bluntly, that they don’t need your votes. They’re living in gerrymandered districts that give them too much power, and they’re winning the elections that matter. If you want to change the broken system in fundamental ways, you need to convince them to vote for policies that involve giving up some of that power. They can keep things just the way they are, or make them much, much worse, without any assistance from you.

And, yes, it’s unquestionably true that a distressingly large number of those voters are openly racist and probably not persuadable. But the hard-core racist fraction is not 100%, and you wouldn’t need a huge effect to make things better. As I said before, even if 39 out of 40 Trump voters in PA, MI, and WI was a full-on alt-right Twitter frog, flipping the vote of that one decent human being would’ve avoided our current situation. I think that would’ve been worth a little bit of effort to respect their feelings, at least long enough to win their votes.

Yes, that’s messy, and compromised, and leaves some big issues unaddressed. Welcome to politics. It’s not about feelings, on either side, it’s about getting enough votes to win elections.

Rant over, catharsis achieved. Shutting up about politics, now.

Read the whole story
7 days ago
Mountain View, CA
Share this story

Batman AU where everything is the same (dead parents, Bruce grows up to be a hirsute giant), but he contracts with a magical talking bat to dispense justice in a frilly magical girl costume.

1 Share

(still p sure the original artist for this is jiro I APOLOGIZE FOR WHAT I HAVE DONE)

Read the whole story
9 days ago
Mountain View, CA
Share this story

The ACLU Goes to War

1 Comment

The Drone Memos collects important memoranda from the Justice Department, addresses by Barack Obama, John Brennan, Harold Koh, Eric Holder and others, executive orders and various documents. It provides an indispensable compendium for anyone interested in the Obama’s administration’s handling of counterterrorism policy and would be an excellent Reader for a seminar. The book is not so much about drones as about targeted killing and its justification. Some of these documents were pried loose from the government by FOIA requests and lawsuits, so it is particularly fitting that a lengthy introduction has been provided by Jameel Jaffer, the former deputy legal director of the ACLU.

That introduction is worth the price of the book alone, for it gives us an articulate and comprehensive critique of administration policy while also providing a memorable self-portrait of the ACLU lawyer as a combatant in the endless struggle against the US government.

Jaffer gives us a remarkable account of how a cerebral and cautious, left-of-center law professor ended up accelerating the targeted killing program from 50 drone strikes in the Bush 43 presidency that killed 296 terrorists, to 506 strikes that have killed 2,040 terrorist (as of January 2016).   What Jaffer doesn’t provide is the reason why this happened; or rather, the reason he does give is so institutionally self-referential as to cast doubt on his grasp of the problem of terror in contemporary strategic affairs.

On Jaffer’s view, the history of the Obama administration’s terror policies is essentially an account of the tortured course of litigation. Indeed litigation provides the all-embracing gestalt of his analysis. Even Jaffer’s metaphors are drawn from trials (the targeted killing of al Awlaki “was a bizarre death penalty case”). Litigation wholly encompasses the Constitution (“if the Constitution meant anything at all, it surely meant that the government could not kill its own citizens without ever justifying its actions to a court”).  The government’s efforts to protect its military-intelligence operations (“they declined to supply on-the- record accounts of the strike and they withheld the results of their post-strike investigation. They control most of the information and disclosed only what they chose to,”) are made to sound like Brady material transgressions. Indeed it is as if the entire rationale of the government’s counterterrorism strategy was concocted to thwart the ACLU’s litigation strategy: “in fact the claim that the United States was engaged in a borderless war against terrorist groups was foundational to the Obama administration’s defense of the drone campaign. It was this claim that permitted the Obama administration to contend that drone strikes — even those carried out away from the zone of actual hostilities — were governed not by human rights laws, which bars the use of lethal force except in very limited circumstances, but by the laws of war, which are more permissive of state violence and less protective of individual rights.”

At a parent conference last week, one of my son’s teachers suggested my wife and I play a card game with Pasha to improve his fine motor skills. She called the game, “Hi-Lo.” After hearing the description, I said, “oh, I used to play that as a child; we called it ‘war.’” “Yes,” she said. “We don’t use that word here.” Apparently Pasha’s teachers and Jameel Jaffer’s colleagues are using the same lexicon, because you would never know from this essay why in the world , “a president committed to ending the abuses associated with the Bush administration’s “war on terror” came to dramatically expand one of the practices [targeted killing] most identified with that war.” That is because Jaffer so studiedly avoids ever acknowledging, much less discussing the consequences of the growth in the threats we face in the wars on terror. You will find nothing in this essay about the massacres in Paris and Brussels, nothing about the rise in sanctuaries in Nigeria, Libya and Yemen, nothing about the deterioration of conditions in Afghanistan and Somalia, nothing about the human rights atrocities in Syria — which should be odd for a human rights advocate — scarcely a mention of the four letter words ISIS and Iraq. It’s rather like saying you will give an account of how a mild-mannered physician ended up sedating and dismembering an innocent man without mentioning the gangrenous leg.

The natural assumption that, because the war is borderless, you want legal authorities to allow you to wage it successfully is turned on its head. Apparently, you invent a war and its parameters, so you can win in court against the ACLU.  But if the war isn’t borderless, and the attacks on New York, London, Paris and Istanbul were not incidents of war, what were they?  Accidents?

Moreover the everything-is-litigation perspective depends upon some fundamental misconceptions about law. Jaffer appreciates that the Obama administration tried to integrate law into its strategic operations — an important, even historic step — but he misconstrues how this is to be done, that is, what in this context lawfulness consists in. He appears to believe that lawfulness is a creation of executive branch lawyers “writing behind closed doors,” but this isn’t quite right. Legal opinions drafted within the executive branch do create doctrine, much as appellate opinions do within the judiciary, but it is misleading to suggest that therefore “the law of the drone campaign had not been enacted by Congress.” Congressional statutes, including appropriation statutes, and joint resolutions are what the legal opinions are construing.

More importantly, the lawyers at the State Department, or at Justice, have no authority to establish legal rules for the executive branch, much less anyone else. The power to make law for the executive branch, either through interpreting a statute or executing the exclusive powers of Article I, or making rules via executive orders, lies solely with the president. It’s not the OLC opinion that provides the law; in fact, it doesn’t even provide the rationale unless the president chooses to adopt it as his own. If the president asserted a rationale for the lawfulness of targeted killing that differed from that proposed by OLC, the OLC opinion would not contravene or override the president’s decision. If he failed to adopt the OLC reasoning as his own, the opinion wouldn’t, like an unsigned but unvetoed statute, become law. The insistent, coercive attempts by litigation to force the publication of clearly privileged legal opinions thus aimed at the wrong target: it was up to the president to present his rationale to the public regardless of whether he accepted the advice of his lawyers.

Jaffer professes to be baffled by what that rationale could possibly be. “For decades the US government had condemned targeted killings, characterizing them as assassinations or extrajudicial executions. On its face, the drone campaign signified a dramatic departure from that position… It was far from obvious what distinguished American drone strikes from the targeted killings the US had historically rejected as unlawful. Nor was it clear how these targeted killings could be reconciled with international human rights law, with a decades old executive order that bans assassinations, with the constitutional guarantee of due process, or, for that matter, with domestic laws that criminalize murder.” The obvious answer to these puzzles, I should’ve thought, is that we are at war and that Congress has authorized that war. It’s not drone technology that makes targeted killing lawful (nothing “distinguishes drone strikes,” per se), it’s the fact—the legal fact–of war.

And not merely this inconvenient fact intrudes on the hermetically sealed chambers of the human rights litigator; it is also the case that we are in a particular kind of war. Like so many others, Jaffer prefers to use shudder quotation marks when he refers to the “war on terror” in order to emphasize its spurious nature.  The wars on terror have done for shudder quotes what the 1970s did for bell bottomed trousers and disco music (we shall see if similar regret eventually ensues). The wars we are currently engaged in don’t seem to fit Jaffer’s conception of a proper war. He is appalled that, “the US was carrying out lethal strikes not only on ‘actual’ battlefields”– – note the shudder quotation marks again – – “but in places far removed from them as well.” This may sound pretty shocking until you think about warfare and the kinds of war that are fought by modern states. In the wars of the industrial nation states for example total war was often waged far from the “actual” battlefield. Remember the burning of Atlanta? The strategic bombing campaigns of World War II? Hamburg? Hiroshima? Now, in the era of wars on terror which is bringing into being the informational market state, targets are likelier to be more discrete but also more dispersed; more individuated, but more global.

In Jaffer’s hands, however, these developments are reduced to a clever if nefarious administration litigation tactic. Jaffer never displays any awareness of a genuine national security threat or context that might be driving legal policy, straining to concede only that it might be that “perceived counterterrorism” imperatives were derailing the constitutional processes of state.

I imagine this is why he uses innuendo to discredit Harold Koh, the former legal advisor to the Department of State. First, Jaffer intimates that Koh was hired by the administration and deployed as a spokesman for its policies because this would discombobulate the human rights community of which Koh was such a prominent and esteemed member. Then, Jaffer implies that Koh reversed his lifelong commitment to human rights by defending US government policy on targeted killing. The very idea that Koh might have had some good reason to believe he was vindicating his human rights ideals when he sought a legal rationale to resist terror apparently is too preposterous to consider.

Perhaps that’s because Jaffer does not concede that global developments should drive legal concepts. With the finality of a barrister eliciting the clinching judgment of an expert witness, Jaffer writes that, “the ICRC does not share the view that a conflict of global dimensions is or has been taking place.” Well, that ought to settle it, right?

There is a good deal of this sort of ipse dixit-ing in the essay, but given Jaffer’s manifest abilities and the subtlety of his approach, one wonders why. Why for example does he repeatedly observe, in mild but ominous tones, that, “secret law is unsettling.”  Of course it is, but OLC opinions are not law and presidential decisions are rarely secret for very long. Indeed one reason the Administration was anxious to transfer targeted killing operations out of CIA was so that the operations could be more fully described to the public.  Jaffer quotes extensively from the “Playbook,” the 18 page set of rules formally known as the Presidential Policy Guidance issued by the president in May 2013. These procedures had been previously reported, relying on unattributed accounts by current and former officials. Under the Playbook, the United States would only use lethal force against individuals who posed a “continuing, imminent threat to US persons,” when there was “near certainty” that the target was present at the site and “near certainty” that noncombatants would not be injured or killed. The government would only use lethal force if it concluded that capture was “not feasible,” and the country in whose territory the strike was to take place was unable or unwilling to address the threat itself; and finally that there were no other reasonable alternatives to stop the threat.” Although the complete Presidential Policy Guidance was not declassified until August 2016, these rules were given a public release by the White House in a fact sheet in May 2013.

There are certainly a number of ambiguities in this description of a bureaucratic process, but what’s the secret law here? I’ve been puzzled as to why the ACLU and others were so eaten up with getting the legal opinions in these matters released – – even holding up David Barron’s confirmation for the First Circuit until the opinions were made public— and apart from the usual arm wrestling for dominance I have come up with just one speculation. It may be that, like the lawsuits that sought to expose those telecommunications corporations that cooperated with NSA, the purpose behind this agitation is mainly to intimidate – – to place in the public record the names of otherwise unknown persons as figures  inimical to our constitutional rights. The officials of the state are the enemies of the people.

If my speculation is true, those are pretty rough tactics but not very different from the larger strategy of delegitimating government action of which we have seen a good deal lately. Jaffer often mentions the alliance between left litigation and right libertarianism, a collaboration of Glenn Greenwald with Rand Paul, an embrace that must make both sides uncomfortable at some level (it certainly does me). Apart from their objectives, they do share a singularly annoying habit of repeating exaggerated claims: “having the executive being the prosecutor, judge, the jury and the executioner all in one is very contrary to the traditions and the laws of this country,” to say nothing of also being contrary to the facts of targeted killing in warfare. “The president asserting that he has the right to kill an American citizen without due process is troublesome,” as it should be, except that no president claims that the process that is due in warfare is the same as that for the commission of a common crime. These claims are reminiscent of Rand Paul’s assertion on the floor the Senate that the NSA was listening in on the cell phone conversations of practically everyone in America. Since last Tuesday, we’ve been living with the consequences of the relentless campaign of delegitimation based on the repetition of sensational charges and the view held by its advocates that the US government is really the chief threat to our liberties and well-being.

Perhaps it’s just irresistible for such a forceful and persuasive advocate to write one-sidedly: the habits of advocacy don’t cultivate evenhandedness. Thus we are given a sympathetic account of al Awlaki’s radicalization, and Jaffer’s horror that al Awlaki’s name was put on a target list, but nowhere are we told that al Awlaki, an American citizen, could’ve walked into any US consulate and turned himself in. He could’ve been met there by an ACLU lawyer to ensure his safe conduct (and his refusal to be interrogated.)

Similarly, Jaffer invokes respected national security figures like John Bellinger, Saxby Chambliss, and Stanley McChrystal who note the potential shortcomings of the targeted killing program –– it alienates local populations, it sacrifices the possibility of intelligence interrogation. But having worked with or known all of these men, I would be quite surprised if they thought the drone programs were unlawful or even of questionable value. This aspect of the essay makes it read more like a brief – – a carefully tailored presentation of sympathetic elements.

One might think that Jaffer would applaud the creation of an experienced cadre of lawyers and intelligence officials to administer the Obama Playbook, but he actually sees this development in a sinister way: there he finds, at last, the source of the true motivation to craft new rules to fight the wars on terror lawfully. It turns out that this motivation does not arise in response to the threats we face, as one might have thought, but from the inertial force of the bureaucracy we have created to cope with those threats— that is the real driver behind the reform of the laws regulating the practices of warfare. “Eight years ago the targeted killing campaign required a legal and bureaucratic infrastructure, but now that infrastructure will demand a targeted killing campaign.”

This is consistent with his claim that unadjudicated law is, “law without limits.” I don’t know what Jaffer would do with constitutional cases like impeachment, the standards for the Senate confirmation of nominees to the Supreme Court, the legality of the Louisiana purchase (to say nothing of secession), the authority of President Truman to send forces to Korea to enforce a UN mandate, and the vast number of unadjudicated – – even nonjusticiable – – but highly significant constitutional cases that form an important part of the corpus of American constitutional law. At one point he seems to concede that due process does not invariably mean judicial process. “There are contexts in which domestic and international law permit the government to deprive individuals of their rights, and even their lives, without first presenting evidence to a judge.… No one proposes that police officers should submit applications to judges before responding even with lethal force to threats they reasonably believed to be imminent.” But Jaffer quickly makes it clear that the essential element in such instances is imminence:  “except on actual battlefields, ‘imminence’ marks the line between situations in which lethal force can be used without prior judicial approval in situations in which it cannot be.”

There are some substantial prudential weaknesses to such a rule. As the Obama administration has asserted, the legal requirement of imminence, as formulated by Webster in The Caroline, is that the action to be deflected permits no time for prevention by any other means. In the case of the terror network that plans its operations in secret for months, even years, it is quite unclear just when the moment arrives when it is too late. Like other catastrophes, nothing much happens for a long time and then things happen very quickly. Jaffer thinks that the months spent in careful intelligence collection and assessment by the government (“many strikes were preceded by bureaucratic deliberation”) simply show that there’s always time for judicial intervention.  Perhaps, he suggests unswervingly, we should consider setting up a “drone court.”

But apart from the prudential considerations, there is a formidable structural issue with such a solution. Jaffer seems to believe that the political question doctrine arises solely from the impracticality of using judges as administrators and adjuncts to the other branches, and there is something to that. The source of the doctrine, however, lies in the structure of the Constitution, not in its practical application: courts will not intervene to decide cases when the controversy at issue is one that the Constitution has allocated to another branch.  For example, the Supreme Court has held that lawsuits challenging Congress’s procedures for impeachment present a political question because the trial and conviction of a president are specifically allocated to the Congress, see Nixon v. United States, 506 U.S. 224 (1993).    Jaffer conscientiously presents the reader with the adverse holdings the federal courts have handed him in the drone litigation, but he is less willing to address, rather than describe with astonishment, why his arguments have been uniformly rejected even by courts with sympathetic judges. If there is a good answer even to the strong case he makes for after-action judicial review of a targeted killing—when the practical pressures for a decision are moot– that answer lies in the fundamental structure of our constitutional government.

The Drone Memos is a superb collection of materials on targeted killing. For persons interested in drone warfare, there probably are better books and drones are an interesting subject in their own right, foreshadowing the problem for states that arises when lethal surveillance technologies become widespread through the commodification of surveillance and weapons technologies.  The informational market state will treat information as the source of war matériel just as the industrial nation state turned to its industrial plant. A market state that treats its citizens as individuated consumers will have to cope with the fact that the bonds of loyalty and deference to the state are frayed when their interaction is just a matter of making transactions. Anyone interested in the subject of drones should read Drone Wars: Transforming Conflict, Law and Policy, edited by Peter Bergen and Daniel Rothenberg.

Rather this book, and its introductory essay, are about the decision to kill a suspected or confirmed militant, often distant from the leading edge of the battlefield and often by remotely manned aerial vehicles. Although this book raises questions about the morality, wisdom and lawfulness of targeted killing, none of these questions can be fruitfully addressed in the absence of some serious discussion about the strategic context in which these decisions are made, the evolution of the global threat, and the changing modes of warfare that are shaping the nature of the state.

Uniting the legal and the strategic is the hallmark of warfare in the 21st century. Anyone interested in the problem might want to spend a couple of hours viewing “Eye in the Sky,” with the estimable Helen Mirren as a UK-based military officer who remotely commands a drone operation to stop a terrorist attack in Kenya.  Art, it seems, can better reflect reality than a depiction of the world , however well-rendered, blissfully unconnected to the evolving geostrategic conflict outside the courtroom.

[Editor’s Note: Watch this space. We will publish a response from Jameel Jaffer after the weekend. And, if you are interested in reading more, see Ryan Goodman, 10 Questions to Ask Yourself When Reading Jameel Jaffer’s “The Drone Memos,” and Jameel Jaffer, Donald Trump’s Drone War]

Image: With permission from Tom Kleh

Read on Just Security »
Read the whole story
13 days ago
Interesting exercise in not seeing the forest for the trees: I expect Jaffer's starting premise to have been something along the lines of _the war itself is illegitimate_ (and therefore justifies nothing). The reviewer appears not to be aware of this even as a logical possibility, even though it's, AFAIK, the _default_ starting premise on the American "left".
Mountain View, CA
Share this story

A Rebuttal For Python 3

2 Comments and 4 Shares

Zed Shaw, of Learn Python the Hard Way fame, has now written The Case Against Python 3.

I’m not involved with core Python development. The only skin I have in this game is that I like Python 3. It’s a good language. And one of the big factors I’ve seen slowing its adoption is that respected people in the Python community keep grouching about it. I’ve had multiple newcomers tell me they have the impression that Python 3 is some kind of unusable disaster, though they don’t know exactly why; it’s just something they hear from people who sound like they know what they’re talking about. Then they actually use the language, and it’s fine.

I’m sad to see the Python community needlessly sabotage itself, but Zed’s contribution is beyond the pale. It’s not just making a big deal about changed details that won’t affect most beginners; it’s complete and utter nonsense, on a platform aimed at people who can’t yet recognize it as nonsense. I am so mad.

The Case Against Python 3

I give two sets of reasons as I see them now. One for total beginners, and another for people who are more knowledgeable about programming.

Just to note: the two sets of reasons are largely the same ideas presented differently, so I’ll just weave them together below.

The first section attempts to explain the case against starting with Python 3 in non-technical terms so a beginner can make up their own mind without being influenced by propaganda or social pressure.

Having already read through this once, this sentence really stands out to me. The author of a book many beginners read to learn Python in the first place is providing a number of reasons (some outright fabricated) not to use Python 3, often in terms beginners are ill-equipped to evaluate, but believes this is a defense against propaganda or social pressure.

The Most Important Reason

Before getting into the main technical reasons I would like to discuss the one most important social reason for why you should not use Python 3 as a beginner:


Python 3’s adoption is really only at about 30% whenever there is an attempt to measure it.

Wait, really? Wow, that’s fantastic.

I mean, it would probably be higher if the most popular beginner resources were actually teaching Python 3, but you know.

Nobody is all that interested in finding out what the real complete adoption is, despite there being fairly simple ways to gather metrics on the adoption.

This accusatory sentence conspicuously neglects to mention what these fairly simple ways are, a pattern that repeats throughout. The trouble is that it’s hard to even define what “adoption” means — I write all my code in Python 3 now, but veekun is still Python 2 because it’s in maintenance mode, so what does that say about adoption? You could look at PyPI download stats, but those are thrown way off by caches and system package managers. You could look at downloads from the Python website, but a great deal of Python is written and used on Unix-likes, where Python itself is either bundled or installed from the package manager.

It’s as simple as that. If you learn Python 2, then you can still work with all the legacy Python 2 code in existence until Python dies or you (hopefully) move on. But if you learn Python 3 then your future is very uncertain. You could really be learning a dead language and end up having to learn Python 2 anyway.

You could use Python 2, until it dies… or you could use Python 3, which might die. What a choice.

By some definitions, Python 2 is already dead — it will not see another major release, only security fixes. Python 3 is still actively developed, and its seventh major release is next month. It even contains a new feature that Zed later mentions he prefers to Python 2’s offerings.

It may shock you to learn that I know both Python 2 and Python 3. Amazingly, two versions of the same language are much more similar than they are different. If you learned Python 3 and then a wizard cast a spell that made it vanish from the face of the earth, you’d just have to spend half an hour reading up on what had changed from Python 2.

Also, it’s been over a decade, maybe even multiple decades, and Python 3 still isn’t above about 30% in adoption. Even among the sciences where Python 3 is touted as a “success” it’s still only around 25-30% adoption. After that long it’s time to admit defeat and come up with a new plan.

Python 3.0 came out in 2008. The first couple releases ironed out some compatibility and API problems, so it didn’t start to gain much traction until Python 3.2 came out in 2011. Hell, Python 2.0 came out in 2000, so even Python 2 isn’t multiple decades old. It would be great if this trusted beginner reference could take two seconds to check details like this before using them to scaremonger.

The big early problem was library compatibility: it’s hard to justify switching to a new version of the language if none of the libraries work. Libraries could only port once their own dependencies had ported, of course, and it took a couple years to figure out the best way to maintain compatibility with both Python 2 and Python 3. I’d say we only really hit critical mass a few years ago — for instance, Django didn’t support Python 3 until 2013 — in which case that 30% is nothing to sneeze at.

There are more reasons beyond just the uncertain future of Python 3 even decades later.

In one paragraph, we’ve gone from “maybe even multiple decades” to just “decades”, which is a funny way to spell “eight years”.

Not In Your Best Interests

The Python project’s efforts to convince you to start with Python 3 are not in your best interest, but, rather, are only in the best interests of the Python project.

It’s bad, you see, for the Python project to want people to use the work it produced.

Anyway, please buy Zed Shaw’s book.

Anyway, please pledge to my Patreon.

Ultimately though, if Python 3 were good they wouldn’t need to do any convincing to get you to use it. It would just naturally work for you and you wouldn’t have any problems. Instead, there are serious issues with Python 3 for beginners, and rather than fix those issues the Python project uses propaganda, social pressure, and marketing to convince you to use it. In the world of technology using marketing and propaganda is immediately a sign that the technology is defective in some obvious way.

This use of social pressure and propaganda to convince you to use Python 3 despite its problems, in an attempt to benefit the Python project, is morally unconscionable to me.

Ten paragraphs in, Zed is telling me that I should be suspicious of anything that relies on marketing and propaganda. Meanwhile, there has yet to be a single concrete reason why Python 3 is bad for beginners — just several flat-out incorrect assertions and a lot of handwaving about how inexplicably nefarious the Python core developers are. You know, the same people who made Python 2. But they weren’t evil then, I guess.

You Should Be Able to Run 2 and 3

In the programming language theory there is this basic requirement that, given a “complete” programming language, I can run any other programming language. In the world of Java I’m able to run Ruby, Java, C++, C, and Lua all at the same time. In the world of Microsoft I can run F#, C#, C++, and Python all at the same time. This isn’t just a theoretical thing. There is solid math behind it. Math that is truly the foundation of computer science.

The fact that you can’t run Python 2 and Python 3 at the same time is purely a social and technical decision that the Python project made with no basis in mathematical reality. This means you are working with a purposefully broken platform when you use Python 3, and I personally can’t condone teaching people to use something that is fundamentally broken.

The programmer-oriented section makes clear that the solid math being referred to is Turing-completeness — the section is even titled “Python 3 Is Not Turing Complete”.

First, notice a rhetorical trick here. You can run Ruby, Java, C++, etc. at the same time, so why not Python 2 and Python 3?

But can you run Java and C# at the same time? (I’m sure someone has done this, but it’s certainly much less popular than something like Jython or IronPython.)

Can you run Ruby 1.8 and Ruby 2.3 at the same time? Ah, no, so I guess Ruby 2.3 is fundamentally and purposefully broken.

Can you run Lua 5.1 and 5.3 at the same time? Lua is a spectacular example, because Lua 5.2 made a breaking change to how the details of scope work, and it’s led to a situation where a lot of programs that embed Lua haven’t bothered upgrading from Lua 5.1. Was Lua 5.2 some kind of dark plot to deliberately break the language? No, it’s just slightly more inconvenient than expected for people to upgrade.

Anyway, as for Turing machines:

In computer science a fundamental law is that if I have one Turing Machine I can build any other Turing Machine. If I have COBOL then I can bootstrap a compiler for FORTRAN (as disgusting as that might be). If I have FORTH, then I can build an interpreter for Ruby. This also applies to bytecodes for CPUs. If I have a Turing Complete bytecode then I can create a compiler for any language. The rule then can be extended even further to say that if I cannot create another Turing Machine in your language, then your language cannot be Turing Complete. If I can’t use your language to write a compiler or interpreter for any other language then your language is not Turing Complete.

Yes, this is true.

Currently you cannot run Python 2 inside the Python 3 virtual machine. Since I cannot, that means Python 3 is not Turing Complete and should not be used by anyone.

And this is completely asinine. Worse, it’s flat-out dishonest, and relies on another rhetorical trick. You only “cannot” run Python 2 inside the Python 3 VM because no one has written a Python 2 interpreter in Python 3. The “cannot” is not a mathematical impossibility; it’s a simple matter of the code not having been written. Or perhaps it has, but no one cares anyway, because it would be comically and unusably slow.

I assume this was meant to be sarcastic on some level, since it’s followed by a big blue box that seems unsure about whether to double down or reverse course. But I can’t tell why it was even brought up, because it has absolutely nothing to do with Zed’s true complaint, which is that Python 2 and Python 3 do not coexist within a single environment. Implementing language X using language Y does not mean that X and Y can now be used together seamlessly.

The canonical Python release is written in C (just like with Ruby or Lua), but you can’t just dump a bunch of C code into a Python (or Ruby or Lua) file and expect it to work. You can talk to C from Python and vice versa, but defining how they communicate is a bit of a pain in the ass and requires some level of setup.

I’ll get into this some more shortly.

No Working Translator

Python 3 comes with a tool called 2to3 which is supposed to take Python 2 code and translate it to Python 3 code.

I should point out right off the bat that this is not actually what you want to use most of the time, because you probably want to translate your Python 2 code to Python 2/3 code. 2to3 produces code that most likely will not work on Python 2. Other tools exist to help you port more conservatively.

Translating one programming language into another is a solidly researched topic with solid math behind it. There are translators that convert any number of languages into JavaScript, C, C++, Java, and many times you have no idea the translation is being done. In addition to this, one of the first steps when implementing a new language is to convert the new language into an existing language (like C) so you don’t have to write a full compiler. Translation is a fully solved problem.

This is completely fucking ludicrous. Translating one programming language to another is a common task, though “fully solved” sounds mighty questionable. But do you know what the results look like?

I found a project called “Transcrypt”, which puts Python in the browser by “translating” it to JavaScript. I’ve never used or heard of this before; I just googled for something to convert Python to JavaScript. Here’s their first sample, a demo using jQuery:

def start ():
    def changeColors ():
        for div in S__divs:
            S (div) .css ({
                'color': 'rgb({},{},{})'.format (* [int (256 * Math.random ()) for i in range (3)]),

    S__divs = S ('div')
    changeColors ()
    window.setInterval (changeColors, 500)

And here’s the JavaScript code it compiles to:

(function () {
    var start = function () {
        var changeColors = function () {
            var __iterable0__ = $divs;
            for (var __index0__ = 0; __index0__ < __iterable0__.length; __index0__++) {
                var div = __iterable0__ [__index0__];
                $ (div).css (dict ({'color': 'rgb({},{},{})'.format.apply (null, function () {
                    var __accu0__ = [];
                    for (var i = 0; i < 3; i++) {
                        __accu0__.append (int (256 * Math.random ()));
                    return __accu0__;
                } ())}));
        var $divs = $ ('div');
        changeColors ();
        window.setInterval (changeColors, 500);
    __pragma__ ('<all>')
        __all__.start = start;
    __pragma__ ('</all>')
}) ();

Well, not quite. That’s actually just a small piece at the end of the full 1861-line file.

You may notice that the emitted JavaScript effectively has to emulate the Python for loop, because JavaScript doesn’t have anything that works exactly the same way. And this is a basic, common language feature translated between two languages in the same general family! Imagine how your code would look if you relied on gritty details of how classes are implemented.

Is this what you want 2to3 to do to your code?

Even if something has been proven to be mathematically possible, that doesn’t mean it’s easy, and it doesn’t mean the results will be pretty (or fast).

The 2to3 translator fails on about 15% of the code it attempts, and does a poor job of translating the code it can handle. The motivations for this are unclear, but keep in mind that a group of people who claim to be programming language experts can’t write a reliable translator from one version of their own language to another. This is also a cause of their porting problems, which adds up to more evidence Python 3’s future is uncertain.

Writing a translator from one language to another is a fully proven and fundamental piece of computer science. Yet, the 2to3 translator cannot translate code 100%. In my own tests it is only about 85% effective, leaving a large amount of code to translate manually. Given that translation is a solved problem this seems to be a decision bordering on malice rather than incredible incompetence.

The programmer-oriented section doubles down on this idea with a title of “Purposefully Crippled 2to3 Translator” — again, accusing the Python project of sabotaging everyone. That doesn’t even make sense; if their goal is to make everyone use Python 3 at any cost, why would they deliberately break their tool that reduces the amount of Python 2 code and increases the amount of Python 3 code?

2to3 sucks because its job is hard. Python is dynamically typed. If it sees d.iteritems(), it might want to change that to d.items(), as it’s called in Python 3 — but it can’t always be sure that d is actually a dict. If d is some user-defined type, renaming the method is wrong.

But hey, Turing-completeness, right? It must be mathematically possible. And it is! As long as you’re willing to see this:

for key, value in d.iteritems():

Get translated to this:

__d = d
for key, value in (__d.items() if isinstance(__d, dict) else __d.iteritems()):

Would Zed be happier with that, I wonder?

The JVM and CLR Prove It's Pointless

Yet, for some reason, the Python 3 virtual machine can’t run Python 2? Despite the solidly established mathematics disproving this, the countless examples of running one crazy language inside a Russian doll cascade of other crazy languages, and huge number of languages that can coexist in nearly every other virtual machine? That makes no sense.

This, finally, is the real complaint. It’s not a bad one, and it comes up sometimes, but… it’s not this easy.

The Python 3 VM is fairly similar to the Python 2 VM. The problem isn’t the VM, but the core language constructs and standard library.

Consider: what happens when a Python 2 old-style class instance gets passed into Python 3, which has no such concept? It seems like a value would have to always have the semantics of the language version it came from — that’s how languages usually coexist on the same VM, anyway.

Now, I’m using Python 3, and I load some library written for Python 2. I call a Python 2 function that deals with bytestrings, and I pass it a Python 3 bytestring. Oh no! It breaks because Python 3 bytestrings iterate as integers, whereas the Python 2 library expects them to iterate as characters.

Okay, well, no big deal, you say. Maybe Python 2 libraries just need to be updated to work either way, before they can be used with Python 3.

But that’s exactly the situation we’re in right now. Syntax changes are trivially fixed by 2to3 and similar tools. It’s libraries that cause the subtler issues.

The same applies the other way, too. I write Python 3 code, and it gets an int from some Python 2 library. I try to use the .to_bytes method on it, but that doesn’t exist on Python 2 integers. So my Python 3 code, written and intended purely for Python 3, now has to deal with Python 2 integers as well.

Perhaps “primitive” types should convert automatically, on the boundary? Okay, sure. What about the Python 2 buffer type, which is C-backed and replaced by memoryview in Python 3?

Or how about this very fundamental problem: names of methods and other attributes are str in both versions, but that means they’re bytestrings in Python 2 and text in Python 3. If you’re in Python 3 land, and you call on a Python 2 object, what happens? Python 3 wants a method with the text name foo, but Python 2 wants a method with the bytes name foo. Text and bytes are not implicitly convertible in Python 3. So does it error? Somehow work anyway? What about the other way around?

What about the standard library, which has had a number of improvements in Python 3 that don’t or can’t exist in Python 2? Should Python ship two entire separate copies of its standard library? What about modules like logging, which rely on global state? Does Python 2 and Python 3 code need to set up logging separately within the same process?

There are no good solutions here. The language would double in size and complexity, and you’d still end up with a mess at least as bad as the one we have now when values leak from one version into the other.

We either have two situations here:

  1. Python 3 has been purposefully crippled to prevent Python 2’s execution alongside Python 3 for someone’s professional or ideological gain.
  2. Python 3 cannot run Python 2 due to simple incompetence on the part of the Python project.

I can think of a third.

Difficult To Use Strings

The strings in Python 3 are very difficult to use for beginners. In an attempt to make their strings more “international” they turned them into difficult to use types with poor error messages.

Why is “international” in scare quotes?

Every time you attempt to deal with characters in your programs you’ll have to understand the difference between byte sequences and Unicode strings.

Given that I’m reading part of a book teaching Python, this would be a perfect opportunity to drive this point home by saying “Look! Running exercise N in Python 3 doesn’t work.” Exercise 1, at least, works fine for me with a little extra sprinkle of parentheses:

print("Hello World!")
print("Hello Again")
print("I like typing this.")
print("This is fun.")
print('Yay! Printing.')
print("I'd much rather you 'not'.")
print('I "said" do not touch this.')

Contrast with the actual content of that exercise — at the bottom is a big red warning box telling people from “another country” (relative to where?) that if they get errors about ASCII encodings, they should put an unexplained magical incantation at the top of their scripts to fix “Unicode UTF-8”, whatever that is. I wonder if Zed has read his own book.

Don’t know what that is? Exactly.

If only there were a book that could explain it to beginners in more depth than “you have to fix this if you’re foreign”.

The Python project took a language that is very forgiving to beginners and mostly “just works” and implemented strings that require you to constantly know what type of string they are. Worst of all, when you get an error with strings (which is very often) you get an error message that doesn’t tell you what variable names you need to fix.

The complaint is that this happens in Python 3, whereas it’s accepted in Python 2:

>>> b"hello" + "hello"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't concat bytes to str

The programmer section is called “Statically Typed Strings”. But this is not static typing. That’s strong typing, a property that sets Python’s type system apart from languages like JavaScript. It’s usually considered a good thing, because the alternative is to silently produce nonsense in some cases, and then that nonsense propagates through your program and is hard to track down when it finally causes problems.

If they’re going to require beginners to struggle with the difference between bytes and Unicode the least they could do is tell people what variables are bytes and what variables are strings.

That would be nice, but it’s not like this is a new problem. Try this in Python 2.

>>> 3 + "hello"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

How would Python even report this error when I used literals instead of variables? How could custom types hook into such a thing? Error messages are hard.

By the way, did you know that several error messages are much improved in Python 3? Python 2 is somewhat notorious for the confusing errors it produces when an argument is missing from a method call, but Python 3 is specific about the problem, which is much friendlier to beginners.

However, when you point out that this is hard to use they try to claim it’s good for you. It is not. It’s simple blustering covering for a poor implementation.

I don’t know what about this is hard. Why do you have a text string and a bytestring in the first place? Why is it okay to refuse adding a number to a string, but not to refuse adding bytes to a string?

Imagine if one of the Python core developers were just getting into Python 2 and messing around.

# -*- coding: utf8 -*-
print "Hi, my name is Łukasz Langa."
print "Hi, my name is Łukasz Langa."[::-1]
Hi, my name is Łukasz Langa.
.agnaL zsaku�� si eman ym ,iH

Good luck figuring out how to fix that.

This isn’t blustering. Bytes are not text; they are binary data that could encode anything. They happen to look like text sometimes, and you can get away with thinking they’re text if you’re not from “another country”, but that mindset will lead you to write code that is wrong. The resulting bugs will be insidious and confusing, and you’ll have a hard time even reasoning about them because it’ll seem like “Unicode text” is somehow a different beast altogether from “ASCII text”.

Exercise 11 mentions at the end that you can use int() to convert a number to an integer. It’s no more complicated to say that you convert bytes to a string using .decode(). It shouldn’t even come up unless you’re explicitly working with binary data, and I don’t see any reading from sockets in LPTHW.

It’s also not statically compiled as strongly as it could be, so you can’t find these kinds of type errors until you run the code.

This comes a scant few paragraphs after “Dynamic typing is what makes Python easy to use and one of the reasons I advocate it for beginners.”

You can’t find any kinds of type errors until you run the code. Welcome to dynamic typing.

Strings are also most frequently received from an external source, such as a network socket, file, or similar input. This means that Python 3’s statically typed strings and lack of static type safety will cause Python 3 applications to crash more often and have more security problems when compared with Python 2.

On the contrary — Python 3 applications should crash less often. The problem with silently converting between bytestrings and text in Python 2 is that it might fail, depending on the contents. "cafe" + u"hello" works fine, but "café" + u"hello" raises a UnicodeDecodeError. Python 2 makes it very easy to write code that appears to work when tested with ASCII data, but later breaks with anything else, even though the values are still the same types. In Python 3, you get an error the first time you try to run such code, regardless of what’s in the actual values. That’s the biggest reason for the change: it improves things from being intermittent value errors to consistent type errors.

More security problems? This is never substantiated, and seems to have been entirely fabricated.

Too Many Formatting Options

In addition to that you will have 3 different formatting options in Python 3.6. That means you’ll have to learn to read and use multiple ways to format strings that are all very different. Not even I, an experienced professional programmer, can easily figure out these new formatting systems or keep up with their changing features.

I don’t know what on earth “keep up with their changing features” is supposed to mean, and Zed doesn’t bother to go into details.

Python 3 has three ways to format strings: % interpolation, str.format(), and the new f"" strings in Python 3.6. The f"" strings use the same syntax as str.format(); the difference is that where str.format() uses numbers or names of keyword arguments, f"" strings just use expressions. Compare:

number = 133

This isn’t “very different”. A frequently-used method is being promoted to syntax.

I really like this new style, and I have no idea why this wasn’t the formatting for Python 3 instead of that stupid .format function. String interpolation is natural for most people and easy to explain.

The problem is that beginner will now how to know all three of these formatting styles, and that’s too many.

I could swear Zed, an experienced professional programmer, just said he couldn’t easily figure out these new formatting systems. Note also that str.format() has existed in Python 2 since Python 2.6 was released in 2008, so I don’t know why Zed said “new formatting systems“, plural.

This is a truly bizarre complaint overall, because the mechanism Zed likes best is the newest one. If Python core had agreed that three mechanisms was too many, we wouldn’t be getting f"" at all.

Even More Versions of Strings

Finally, I’m told there is a new proposal for a string type that is both bytes and Unicode at the same time? That’d be fantastic if this new type brings back the dynamic typing that makes Python easy, but I’m betting it will end up being yet another static type to learn. For that reason I also think beginners should avoid Python 3 until this new “chimera string” is implemented and works reliably in a dynamic way. Until then, you will just be dealing with difficult strings that are statically typed in a dynamically typed language.

I have absolutely no idea what this is referring to, and I can’t find anyone who does. I don’t see any recent PEPs mentioning such a thing, nor anything in the last several months on the python-dev mailing list. I don’t see it in the Python 3.6 release notes.

The closest thing I can think of is the backwards-compatibility shenanigans for PEP 528 and PEP 529 — they switch to the Windows wide-string APIs for console and filesystem encoding, but pretend under the hood that the APIs take UTF-8-encoded bytes to avoid breaking libraries like Twisted. That’s a microscopic detail that should never matter to anyone but authors of Twisted, and is nothing like a new hybrid string type, but otherwise I’m at a loss.

This paragraph really is a perfect summary of the whole article. It speaks vaguely yet authoritatively about something that doesn’t seem to exist, it doesn’t bother actually investigating the thing the entire section talks about, it conjectures that this mysterious feature will be hard just because it’s in Python 3, and it misuses terminology to complain about a fundamental property of Python that’s always existed.

Core Libraries Not Updated

Many of the core libraries included with Python 3 have been rewritten to use Python 3, but have not been updated to use its features. How could they given Python 3’s constant changing status and new features?

What “constant changing status”? The language makes new releases; is that bad? The only mention of “changing” so far was with string formatting, which makes no sense to me, because the only major change has been the addition of syntax that Zed prefers.

There are several libraries that, despite knowing the encoding of data, fail to return proper strings. The worst offender seems to be any libraries dealing with the HTTP protocol, which does indicate the encoding of the underlying byte stream in many cases.

In many cases, yes. Not in all. Some web servers don’t send back an encoding. Some files don’t have an encoding, because they’re images or other binary data. HTML allows the encoding to be given inside the document, instead. urllib has always returned bytes, so it’s not all that unreasonable to keep doing that, rather than… well, I’m not quite sure what this is proposing. Return strings sometimes?

The documentation for urllib.request and http.client both advise using the higher-level Requests library instead, in a prominent yellow box right at the top. Requests has distinct mechanisms for retrieving bytes versus text and is vastly easier to use overall, though I don’t think even it understands reading encodings from HTML. Alas, computers.

Good luck to any beginner figuring out how to install Requests on Python 2 — but thankfully, Python 3 now comes bundled with pip, which makes installing libraries much easier. Contrast with the beginning of exercise 46, which apologizes for how difficult this is to explain, lists four things to install, warns that it will be frustrating, and advises watching a video to help figure it out.

What’s even more idiotic about this is Python has a really good Chardet library for detecting the encoding of byte streams. If Python 3 is supposed to be “batteries included” then fast Chardet should be baked into the core of Python 3’s strings making it cake to translate strings to bytes even if you don’t know the underlying encoding. … Call the function whatever you want, but it’s not magic to guess at the encoding of a byte stream, it’s science. The only reason this isn’t done for you is that the Python project decided that you should be punished for not knowing about bytes vs. Unicode, and their arrogance means you have difficult to use strings.

Guessing at the encoding of a byte stream isn’t so much science as, well, guessing. Guessing means that sometimes you’re wrong. Sometimes that’s what you want, and I’m honestly ambivalent about having chardet in the standard library, but it’s hardly arrogant to not want to include a highly-fallible heuristic in your programming language.

Conclusions and Warnings

I have resisted writing about these problems with Python 3 for 5 versions because I hoped it would become usable for beginners. Each year I would attempt to convert some of my code and write a couple small tests with Python 3 and simply fail. If I couldn’t use Python 3 reliably then there’s no way a total beginner could manage it. So each year I’d attempt it, and fail, and wait until they fix it. I really liked Python and hoped the Python project would drop their stupid stances on usability.

Let us recap the usability problems seen thusfar.

  • You can’t add b"hello" to "hello".
  • TypeErrors are phrased exactly the same as they were in Python 2.
  • The type system is exactly as dynamic as it was in Python 2.
  • There is a new formatting mechanism, using the same syntax as one in Python 2, that Zed prefers over the ones in Python 2.
  • urllib.request doesn’t decode for you, just like in Python 2.
  • 档牡敤㽴 isn’t built in. Oh, sorry, I meant chardet.

Currently, the state of strings is viewed as a Good Thing in the Python community. The fact that you can’t run Python 2 inside Python 3 is seen as a weird kind of tough love. The brainwashing goes so far as to outright deny the mathematics behind language translation and compilation in an attempt to motivate the Python community to brute force convert all Python 2 code.

Which is probably why the Python project focuses on convincing unsuspecting beginners to use Python 3. They don’t have a switching cost, so if you get them to fumble their way through the Python 3 usability problems then you have new converts who don’t know any better. To me this is morally wrong and is simply preying on people to prop up a project that needs a full reset to survive. It means beginners will fail at learning to code not because of their own abilities, but because of Python 3’s difficulty.

Now that we’re towards the end, it’s a good time to say this: Zed Shaw, your behavior here is fucking reprehensible.

Half of what’s written here is irrelevant nonsense backed by a vague appeal to “mathematics”. Instead of having even the shred of humility required to step back and wonder if there are complicating factors beyond whether something is theoretically possible, you have invented a variety of conflicting and malicious motivations to ascribe to the Python project.

It’s fine to criticize Python 3. The string changes force you to think about what you’re doing a little more in some cases, and occasionally that’s a pain in the ass. I absolutely get it.

But you’ve gone out of your way to invent a conspiracy out of whole cloth and promote it on your popular platform aimed at beginners, who won’t know how obviously full of it you are. And why? Because you can’t add b"hello" to "hello"? Are you kidding me? No one can even offer to help you, because instead of examples of real problems you’ve had, you gave two trivial toys and then yelled a lot about how the whole Python project is releasing mind-altering chemicals into the air.

The Python 3 migration has been hard enough. It’s taken a lot of work from a lot of people who’ve given enough of a crap to help Python evolve — to make it better to the best of their judgment and abilities. Now we’re finally, finally at the point where virtually all libraries support Python 3, a few new ones only support Python 3, and Python 3 adoption is starting to take hold among application developers.

And you show up to piss all over it, to propagate this myth that Python 3 is hamstrung to the point of unusability, because if the Great And Wise Zed Shaw can’t figure it out in ten seconds then it must just be impossible.

Fuck you.

Sadly, I doubt this will happen, and instead they’ll just rant about how I don’t know what I’m talking about and I should shut up.

This is because you don’t know what you’re talking about, and you should shut up.

Read the whole story
14 days ago
It took me about two weeks to get used to Python 3(.4, at the time) and now I find it mildly irritating whenever for some reason I have to use version 2. 3 really is a better language and I would argue that it is _easier_ to learn if you're starting from scratch. There's only one kind of object! Lazy iteration is the norm, not something you have to remember to ask for! The standard library is much better organized! You don't have to jump through cryptic hoops to use Unicode text! The only thing I really _miss_ from 2.x is `"6920616d206865726f".decode("hex")` and that's mostly because I was playing a lot of _The Talos Principle_ at the same time as I was trying to get used to v3.
Mountain View, CA
Share this story
1 public comment
14 days ago
As someone new to python, the people fighting against python 3 make me think the python community is a bit insane.
14 days ago
Next Page of Stories