Deep Learning with SAS

I don’t know if you’ve seen SAS‘s campus. It’s a collection of enormous glass buildings. Abstract art greets you throughout the grounds. Outside the S-building stands a thirty-foot structure of red pipes bent at 90 and 45 degree angles and inside is what looks a bit like the cross-section of a cube. Looking out a window, one can just see the top of another big glass building over a copse of conifers.

As of the Friday before last, this is the organization that offers the funds that provide my stipend and pay for my tuition. Leaving the Leonardo project happened so subtly that my team and I all forgot to have some sort of commemoration ceremony. Last Friday I stood up from my desk shook my team leader’s hand, telling him it was “good working with him,” then he said we should arrange for one last team celebration. Our co-workers joked that this might be like the going-away parties in the Godfather, and I recommended we make sure to  eat at a popular, well-lit restaurant.

Now my way is paid to work on deep learning for language. Really, I couldn’t imagine a better fit to my interests. Of course I’m interested in Natural Language Processing, and my zeal for deep learning is such that I need to actively temper it to avoid poisoning conversations by implying to other researchers that all the techniques they’ve been using are outdated and soon to be obsolete. Now I get to work with a group of people to put my money where my mouth is and actually make something revolutionary, or at least useful.

Since we’re just starting, right now I’m reading papers about deep learning language techniques. I’ve found twenty-five papers over the last three years in the small set of conferences that I’ve checked. There’s an awful lot of interest in the domain of machine translation, but my favorite paper thus far has taken a sentiment analysis approach to identifying ideological biases in written text. With deep learning, it is able to understand that “the big lie of ‘the death tax'” is ideologically liberal, whereas an old-style system would take words two or so at a time and likely see “death tax” and think conservative.

I spoke with my new team leader, Brad, about using the big, fancy computer they’ve offered me for my personal research. He said that would not be a good idea, as it would complicate the ownership of whatever research I produced. “If you want a bigger computer,” he said, “I’ve just got one laying around that nobody’s using. I can get that to you within a week.”

Things are going pretty well.


Sam’s Blog Classic: Wolfman

This is a re-post from August 21st, 2011. At this time I had just moved in to North Carolina State University.mypicture

So, I get to NC State, and move into Wolf Village on Wolf Village Way. A statue of a wolf greets me at the door. From Wolf Village I take the Wolfline bus and pick up some Wolfpack T-shirts for my parents. After I get back I need to print an employment eligibility form, so I go down to the computing center and log on to WolfPrint (through the WolfNet service) and attempt to print my document, but a picture of an anthropomorphic wolf with empty pockets informs me that I need to bump up my print quota. Unfortunately, it’s unwise to give credit card information on a public computer, so I have to take my Wolfmobile back to my Wolfcave. Shimmying down the wolfpole I find my wallet next to my wolferang and a few cans of wolf-Shark repellent. All this going back and forth was making me hungry, so as I drove the Wolfmobile back to the computing lab I wolfed down a bag of wolf wolfs and also wolfed the wolf wolf wolf all the way to wolf wolf wolf wolf wolf. Wolf wolf wolf wolf wolf wolf wolf-wolf, wolf wolf wolf. Wolf.

I have now written the word “Wolf” enough that it no longer looks like a real word to me. This is called Jamais Vu, the sense that a familiar experience is somehow unfamiliar. It is the opposite of the more well-known Deja Vu. Presque Vu is the term for something on the tip of one’s tongue that one just barely can’t remember. Of the three it’s the easiest to work into normal conversation, I’ve found.

Anyway, my roommates are really chill, and have presented no issues that have significantly impacted my ability to sleep. I am already feeling really busy, even though I have hardly done anything yet, so I’m not entirely certain what that portends. Mostly I expect I just have to finish getting settled and I’ll be fine.


You are so awesome.


While en wolf route, I recommend several stops: the home of Little Red Riding Hood’s grandmother, the homes of the three little pigs, and that pond (over there!) with the duck in it. This will save you a few wolf bucks because you won’t need to buy as many bags of wolf wolfs. Cheers, Canis Lupis
Who is Canis Lupis???
I must know!
That was me, Sam, by the way.


Ever since Nate Silver of correctly predicted the last presidential election, he’s gotten a bit of a name for himself. Now he’s apparently the face of election polling. All of the activist emails that I get have been saying things like “Nate Silver says we can’t do it. He says we’re going to lose. Let’s prove Nate Silver wrong.” I’ve been getting a lot of these emails.

Then my friend got a job canvassing for Kay Hagan, incumbent senatorial candidate for North Carolina. From what I understand, the below Dilbert comic roughly represents how his job works.


That’s not fair. He’s allowed to get help from friends as well as family. Now I’m out canvassing for Kay Hagan in Hillsborough. Not because a boss with devil horns for hair forced me to, but because I personally would rather have Kay Hagan in congress than her competitor, and I like to try and make the things I want to be be the things that are.

I’ve learned a lot in my two canvassing endeavors so far. Mostly that the people on the list to be canvassed are often people who don’t want to talk with canvassers. It’s remarkable how many people don’t answer their doors.  Once a person started to open the door and then abruptly stopped, probably noticing that we weren’t anyone she knew. My father, canvassing with me, also noticed that people often didn’t want to talk with him. There seemed to be a disproportionate number of people in the shower in the middle of the day when a canvasser was at the door.  Once a little girl at the window shouted at her mother “Mommy, I’m hiding upstairs! You can hide with me if you want!” No one answered the door.

Nevertheless, we did get some people willing to talk to us. A lot of people were more invigorated by dislike of Thom Tillis than like of Hagan, so I encouraged them to think of a vote for Hagan as a vote against Tillis. It seemed like someone here and there may have gone from not voting to voting for Hagan. I hope so.

Team Humanity


An article showed up on Digg today, asking if sacrificing for one’s team should be considered altruistic. In the context of sports, the author, David Papineau, raises a question of whether a player sacrificing for his team is doing so for his or her fellow teammates (altruism) or is instead identifying with the abstract concept of a “team,” of which he identifies as a member.

Papineau then suggests an extension to traditional game theory that  allows separate agents to behave as a group. He uses an example of two player-agents in soccer where the correct answer is only clear when both agents act as a unit. You can look at the article for the details, but basically each agent’s ideal action is dependent on the action of the other, but of all the possible combinations of both agents’ actions, there is a clear best outcome.

Papineau then brings up the prisoner’s dilemma, wherein the solution is also trivial when attacked with group, rather than individual reasoning. Papineau notes that group reasoning falls apart when even a very small proportion of members do not behave according to the group reasoning. However, humans very often do behave with group reasoning. Watching The Wire, it’s more than clear that criminal gangs of all stripes have solved the prisoner’s dilemma using group reasoning, for example. The gang, the family, the syndicate, these are all teams on which people play, and with the proper cultural mindset it’s relatively easy to imagine that a player will be confident that his teammates will hold up their end of the bargain.

Now let me take Papineau’s article in a different direction: clearly this propensity for group-based reasoning has allowed humans to prosper through cooperation, but why does it so often fall down after a certain point and leave us with warring tribes instead of a world-wide harmony?

To answer that, here’s another question – what team am I on? The obvious first, I am on my own team. Then I have a series of other teams, in very rough order of closeness to me, my relationship, my work, my family, my friends, North Carolina, America, humanity. My duty to each of these gets more remote and abstract as the entity gets bigger and my place in it gets smaller, and therefore I’m willing to sacrifice less for one group to benefit a group more removed from me.

That may not be the central issue, though. Clearly people can be good at placing a very large entity’s needs above their own – this is what nationalism and its ability to motivate massive armies to kill and die for their nation proves. Nationalism is anything but a given in a nation. It relies on enormous propaganda drives. It can remain strong even when the nation does not fulfill its side of the bargain (read: veteran’s healthcare). Also, one can identify with one’s nation and not with one’s leaders, which is why patriotism can mean so many different things to different people. Nevertheless, the nation appears to be the largest entity so far that has been able to get people to identify as team members and sacrifice for it.

The forces encouraging people to identify with all of humanity do not have enormous propaganda on their side. The complexities of humanity make it difficult for us to all agree on what actions to take, and many of us don’t even agree on what success looks like, making it difficult to form a team mentality. The closest we have are our ethical standards – journalism, science,  human rights, and rule of law are four that come to mind. Instead of identifying with large groups of people, we can identify with sets of rules and values that are designed to remain the same despite the frailties of the people involved. Maybe that’s how Team Humanity should play the game.

Evangelical Atheism


I watched a YouTube video a while back [1]. I’ve put the link at the bottom because it’s pretty offensive. I recommend you read this article before deciding whether to watch it, you won’t need it to understand the article. This video portrays an atheist having a discussion with stereotypical representatives of a gaggle of different religions.

The video’s theme is that each of these religions has a different, conflicting understanding of God, and all-powerful being that he is, if God really had something important to say, he should be able to make it clear enough that there wouldn’t be hundreds of different interpretations of his words.

A second, more subtle point appears to be one of shared humanity. At least, that’s what I read into it the first time I watched it. The second time I wasn’t so sure. “If we can throw off the chains of theism,” it seemed to me to say, “we can realize that we are all one human family.” After the atheist’s impassioned plea and a requisite pause, all the theists return to bickering. The atheist walks away, as if to say, “of course they’ll never listen to reason, they believe in God.”

If we look at the video again without the stirring music, we see a bunch of people arguing, one person shouting at everyone else while they politely listen, and then that one person walking away. Really, the atheist is just bickering along with everyone else. Regardless of whether or not his claims make more sense, the video itself appears to admit that making the argument is fruitless.

Here is my question: for an atheist, what is the purpose of obsessing over the metaphysical? If you believe people are going to go to hell if they don’t believe in your God that’s one thing, but where’s the good deed in convincing people that heaven isn’t real? Especially if you’re extremely unlikely to succeed.

If you really want to make everyone believe what you do, good luck. That’s just the same thing many of those other religions you think are so crazy are trying to do, and they’ve been trying a lot longer than you have. If what you’re really worried about, and I think this is something we should all worry about, is the interference of religion with science and good public policy, then you may have a shot at making a difference. As others have said, a good example of championing rationalism, if not specifically atheism, is Neil Degrasse Tyson’s “Cosmos” series [2].

Instead of attacking Christianity, “Cosmos” presents the universe as it is understood today and the history of discovery behind this understanding. In discussing this history, Tyson doesn’t shy from mentioning when politics, economics, or, yes, religion, gets in the way of human advancement. He also, it is important to note, mentions when Christianity is helpful. Tyson explicitly states, for example, that Michael Faraday, the father of much of our understanding about electromagnetism, was an evangelical christian, and suggests this helped him come to contribute what he has.

So, fellow atheists, I implore you to consider what battle it is you’re fighting. Is it really religion itself that you can’t stand? Is there nothing you prefer to believe that you know really isn’t true? Is belief in the supernatural mutually exclusive to scientific advancement? Is a futile war against all of religion a good use of your time, or is there something better you can do to counter the issues you see in the world? Just something to think about.



Some other interesting links:
A brief discussion of the history of atheism and an argument that it is generally a reaction to oppressive religion.
A channel dedicated to atheism
A brief critique of AtheistTV’s strategy

Post-Privacy America

Imagine, if you will, that the police in Ferguson had been wearing cameras. Instead of conflicting stories about what happened, we would have video evidence making the facts clear. Now imagine that instead of sitting on a Ferguson hard drive, the video was automatically uploaded to the Internet. Everything that Ferguson cops do is on display for the whole world at all times. That would make it more difficult for a cop to do something he’s not supposed to do now, don’t you think? Ok, now let’s say all the cops in the country are constantly monitored at all times when on duty. Now what say we monitor them off duty? What if we monitor all state employees, including politicians. You know what? Let’s make everything everyone does known to everyone else.

At first, it’s chaos. Your neighbor now knows about your unusual taste pornography and is too horrified and ashamed to speak with you again. His wife, though, now knows about his taste in other women, and you take out your popcorn and watch on your computer screen the clip of that holier-than-thou jerk getting kicked out of his house, which it turns out is in the name of his soon to be ex-wife. Your children learn a whole lot very quickly about how the world really is. Not only do they discover a wealth of bad words and your unusual taste in pornography, which is very difficult to explain to them, they use the new surveillance program to find santa-claus and discover that it’s just you. You’re not even wearing a santa suit – just your ratty old “Jingle Bell Rock” sweater. Amid all the crying, no one gets much sleep that night. Spending much of the night trying to explain your unusual taste in pornography to your wife, you’re beginning to get a pretty solid opinion that you don’t like this program of radical honesty.

But by the next day the news reports start coming in. You thought the news would be dead, but it turns out they’re more active than ever – somebody’s got to sort through all this information. Three quarters of the scandals attributed to the president turn out to be unequivocally true, but scandals are streaming in from all over the country so quickly that before you’ve finished your breakfast the president is old news – more than half the state and local politicians in the country are getting attacked on both sides for rampant corruption. It seems like almost everyone in power is using that power against rather than for the American people. Talk begins wondering how we can get rid of all this corruption without the country collapsing. Others wonder how this country hadn’t collapsed already. Already overcrowded jails fill even more as the crimes of those not in power show up on the universal recordings. Suffering upon suffering is shown in vivid color to horrified Americans around the country. Poverty, starvation, homelessness,violence, and myriad other social problems are abruptly impossible to ignore. It is a crisis, but we are a nation of crises, and we respond.

As a nation we decide just to use our existing voting system – with our newly educated voting body – to weed out corruption. Our new politicians know that they will be judged based on their actions rather than their rhetoric and politics becomes much more mature as a result. With advanced video analysis, complete information allows for unambiguous statistics that settle what used to be areas of political contention. Does increased government spending help the economy? How many people who are very poor really need help and how many are just lazy? What actions that people and government have taken really help to reduce the demand for abortions? These questions are now answered by facts instead of stump speeches.

Over time, your children learn to live in the world that is rather than the world that they imagined in their ignorance. Your wife stays with you and, while she never really understands your tastes, decides that they aren’t any worse than any other quirk in your personality and the two of you end up closer than ever in the presence of unprecedented mutual understanding. Your friends that remain with you are true friends. Many of them have lost friends when their own secrets became public or when they discovered the horrible things their friends did and still do. Some mourn the passing of these shallow relationships, some are pleased to know who they can and can’t trust.

As your children grow up in the new society, you notice they have no interest in idle chatter. With no secrets, they grew up on harsh realities and important distinctions and they take interest in improving the world rather than hiding behind the fictions that defined previous generations. Their children grow up thinking of privacy and secrets as an antiquated notion – a bizarre artifact of the past that as hard as they try they can’t quite wrap their minds around why it was valued so highly. As your grandchildren come of age, they ask you why people, even people who were not doing bad things, were so obsessed with keeping secrets. “I don’t know,” you admit, “I guess… I guess we were just afraid.”


I visited a Chinese restaurant with Alice and a friend of mine the other day. This is the restaurant that serves the stinky tofu, for fans of the original Sam’s Blog. I ordered it again, not with the intention of eating it myself, but of feeding it to Alice and seeing her reaction, which, unfortunately, turned out to be disappointing. What’s more the tofu made the whole table smell like horse manure and even when I wasn’t eating it threatened to ruin my meal. Nevertheless, I gritted my teeth and attempted to distract myself with the otherwise excellent food and conversation.

The conversation was, as it turned out, the most interesting part.  My friend, an immigrant from China, told me that she was against the DREAM act because she thought that we should be trying to help the countries the people were fleeing be better places to live instead. Certainly that does seem like a better long-term solution than just having everyone in countries with problems move to the United States, but of course it doesn’t seem particularly applicable in the short term.

My friend mentioned that she, who had come to the United States legally, was not a citizen, while the undocumented immigrants covered in the dream act would become citizens. She mentioned how her work visa renewal got in late and due to this bureaucratic mix-up she cannot get a driver’s license for possibly as many as three months. We had to pick her up to take her to the restaurant.

She went on to describe how China actually has immigration restrictions within its borders. In order to leave the province of your birth, you have to have sponsorship from an employer, for instance. China does this because it is so crowded that population centers like Beijing could not possibly sustain the number of people that would go there if they had free reign to do so, she said.

This raises the question – if everyone who wanted to live in America could just go and live there, what kind of a nation would we have? Could the famous breadbasket of the world support perhaps, let’s conservatively estimate, two billion people? I am fully in favor of housing refugees when their own countries have troubles, but the notion that we should just invite everyone to come live in our country indefinitely may be dangerous. Then again, keeping them out seems selfish and cruel, and making all of those countries great places to live is a tall order. Not to mention, we still haven’t figured out how to help all the people that already are here!  Once again the right thing to do is difficult to determine.


It's about whatever I say it's about