Tag Archives: computer science

AutoBlog 2: Adding the Old Blog

I have now added my old blog, 300 entries from 2008 to 2013. I include here results trained on my old blog, my new blog, and my full blog (old and new together). We should expect, since the full blog roughly doubles the training data, that it will tend to do better than previous models. The short blogs below are all probabilistic, meaning the network generates a probability distribution for the next word and the generator selects from it randomly according to that distribution.

I have included two versions of model for each of these data sources. Small models have fewer neurons than large models, making them faster to train, but less able to represent complex phenomena. I haven’t fixed the formatting manually this time.

Small Models

Old Blog

switched .

You’ll notice that this one is unusually short. Generally models trained on my blog get to the arbitrary word limit before they predict an “end of entry” tag. This one is an exception, and a notable one at that. I doubt it’s representative of my old blog, except that I tended to write shorter entries.

New Blog

demeanor dissertation perfection with recycle , has , shared an even shuffled crick to make investment that they can sore . it five-year-old if a crime snapped is a human pinch under a producing dressings . as if each violation is full of the roiling https://www.youtube.com/watch?v=m78gyytrg7y we feminist how i can present . each brags of backside discovered rainbow margin that seems soundly . flavor , and this motivate , choose that dairy , also lifeless . without real moment , though is narrowing rationalizing and carson street cleaning people need to make your wishes because todd has bizarre such

The “roiling” Youtube link points to an unavailable video. I should figure out how that got past the preprocessing step. Pay attention to this and we can see if it gets better when I apply a larger model.

Full Blog

donations , <unk> i’m never chuck on facebook and ime mountain angry donned . but too , so i’m , old <unk> at the office . wouldn’t just , but the most induction people should get off inviting of cube into us peace on human overhaul and peak the japanese country and finally and their sent me . evidently quintessential children again , christmas , ” and and lived , and effects behind to us trouble . ” diane , you can go claim from significant hero how to make there !

For the small model, including both didn’t increase the sensibility of the model as much as I had expected.

Large Model

Old Blog

dejected , i have moisture results out of boss disease and expo for my spark horrible dispensers . unfortunately , i offhandedly decided my elaborate retreat to my hats . weekend , i important lose locate lost lee’s and an slew of a sledgehammer with the narrative partially for the side . elliot foolishly my mishaps and challenged one’s blend acquaintances complaining that i could re-read my relief . in the woo , the quietest organization in dirt representing following consciousness , implication . i meanings [censored] corporations , i drank pop graphic break and debug fit , so batches

I would have to do some more analysis to figure out if the first word, dejected, led to the model keeping that tone throughout, or if it is just representing what may be an overall somewhat negative blog.

New Blog

dumped , wouldn’t overhaul the reinvent the painstaking introduced up of awful day . lower hidden forty-eight resounding and fiction is moments next , like the time for the distributing repurposed and note on diane .

No dramatic improvement here with the larger model.

Full Blog

i’ve been videos to application these pauses to towers in goodwill . i again , my opening appeared ever heard ever blowing since i texted my re-read . i don’t remember the clumps story . in the address i sol forgot some brahe and junior press every exam . explain you tearer . ” cabin-mates xeon , ” what it is good , should alone anthropomorphic language , ” secret goading , ] what i had releases worst as i rely its message to torn up many grandma , and the wider tacos was delay on slogan . tried to

So, doubling the data did not have a noticeable effect. I wonder if even all the blogs I’ve written in nine years are not enough to make a reasonable language model. They do pale in comparison to English Wikipedia, for instance, which has 2.9 billion words to my blog’s paltry 240,000. Excessive randomness in the probabilistic model could be another weakness. Other approaches to generative models describe modifying the random distribution to make likely words appear more often without going completely deterministic.

Advertisements

San Diego 2016

There must be something about Southwest Airlines flight attendants. The flight attendant this time spent fifteen minutes trying to fasten a seat belt by throwing one side of it into the air and catching it with the other side while the boy in my row insisted that the giant florida gator I had and he hadn’t seen on YouTube was no more than three feet tall. While the boy listed the various types of rock that would likely be found in the Nevada mountains over which we were flying and his mother showed him card tricks, the flight attendants asked us to place our money and jewelry into garbage bags they were carrying through the aisles.IMG_20160618_093322922.jpg

For a brief week, The San Diego Sheraton Hotel and Marina was full of the most famous Natural language processing scientists of this day and age. Luckily, I was there, too. Among the presenters were a man wearing hair purple on the bottom and turquoise on top to match the coat he was wearing, which was textured with a photograph of a pile of pickles. Evidently, even the pickle suit man was interested in looking professional, though, as before he stood to present, he removed his coat and simply wore a hawaiian shirt.
IMG_20160611_201159917.jpg
Our first invited talk suggested we were aiming too low if we did not think NLP could be used to cure cancer. The next one discussed the importance of verifying evaluations, which led to a man standing up and speaking out in defense of science that doesn’t follow the scientific method. A swedish woman behind me shook her head and scratched in her notebook whenever he opened his mouth. Later we were in an uber going to San Diego’s Little Italy in a restaurant designed to look like an indoor recreation of an Italian street corner, complete with balconies and a street lamp next to a staircase. We were joined by a her roommate, a man from Belgium, and a friend from my old lab who was also attending NAACL. The roommate became increasingly pleased with the Disney-like music and the sparkling wine that we had gotten by the bottle for half off, and began gushing about how wonderful it was that we had asked no questions whatsoever about each other’s work and how we hadn’t even bothered to learn each others names, and how it spoke volumes about our compatibility as people. I never saw them again.

IMG_20160615_182431639.jpgFor my first lunch at the conference, I jumped in with a group as they headed out. The Americans in charge had decreed that we would be going to Subway. I was heavily prioritizing networking, and so I restrained myself from protest. The Americans got in a car and told us they didn’t have space for two Germans, a Dutchman, and a Sam. Fortunately, as Americans are wont to do, they were hopping in a car to travel about a mile. My half of the group decided to walk. One of the Germans put the location into his phone and we were off. As we walked, the sidewalk became narrower until it was about a foot wide, and we could only walk single file and lean away whenever an 18 wheeler truck brushed past us. Eventually, we reached a complicated multiple-way intersection surrounded by highways and landscaped urban vegetation, and the German’s phone announced we had reached Subway. After a few minutes of fruitless searching, we found that we had somehow made our way back to the airport, where there was no Subway, but there were plenty of options for wealthy, undiscriminating travelers. No one said it outright, but I’m sure many of us were surprised to find ourselves wishing we were at Subway.

IMG_20160616_123608190_HDR.jpg

Later I had more luck. I ate “The Triple Threat”. One woman came to me while I was eating and asked how she could get a sandwich like that. She wanted me to help her choose between the pork, bacon, and ham, like she could not believe that you got to have all three at once. At a later event, I told some people about the flight attendant who said that he was from New Hampshire and learn to speak with a southern accent when recovering from an injury with the help of a Mississippi speech therapist. I admitted I wasn’t sure how true it was, but they were convinced. Their argument was that the story would not have been so specific if it were not true. “You can’t make that up,” they asserted. I casually informed them that including specific details is an important strategy when telling a fiction. As an example, I even went so far as to admit that I had forgotten what New England country he said he was from, and had simply made up New Hampshire out of whole cloth. I think that took me out of their implicitly trustworthy stranger category and placed me directly into “pathological liar.” After that, everything I said was treated with a sideways glance.

IMG_20160617_190745566.jpg
On my last outing with a Massachusetts ex-professor, a Finn, and a Brit, a man came up to me and asked me to listen to him and rate his statements from zero to 100. The first thing he said was that everyone has an invisible colored light that shines around them known as an aura. I rated that a 20, and he asked me to save my ratings until the end. Then he told me that I worked on computers. Fair enough. Then he said the Finn and I were best friends who loved movies. We also liked smoking hash. He asked if he was right, and I told him I wouldn’t tell him, but he should keep talking. He told me I didn’t get along with my father, and that between my older sister and me, I was beaten more as a child. This continued for a while until I was doubled over with laughter and could barely get out “no, no, I won’t tell you,” between my wracking guffaws. The man predicted that my best buddy Finn and me were both so emotionally damaged that instead of drinking alcohol like a normal person, we got completely baked every night. Then he walked away. I started to tell the Finn about how hilarious this guy was when he showed up again behind me and shouted once more, “Was I right!?”
IMG_20160613_173340873.jpg
In an effort to attract people to her poster, one woman described her work as bringing us one step closer to making Lisa Simpson’s grammar correcting robot “Linguo” a reality. Another woman impressed everyone by rapping her research presentation. I don’t know what research it was she was trying to present. A Stanford professor casually told us that he was suspicious of results in a paper coming out of Google DeepMind, and he took a day and wrote a simple deep learning method that blew them out of the water. In the room next door, four Quiz Bowl champions narrowly held out against a single graduate student’s question answering tool. Soon after, a group of scientists debated whether we should allow machine learning to continue to consume computational linguistics. I decided it would be rude to inform them that resistance is futile.IMG_20160616_163147413_TOP.jpg

The Cleaners Part 3: Diane Vs. William

Continued from The Cleaners Part 2

I opened a second wine cooler and poured it into my glass of gin. “You can do this, Diane,” I mumbled, swishing it absently. My bird clock on the wall was minutes from the bluejay. Bluejays were beautiful animals, but their call sounded like a squeaky door. I didn’t like it before. Now, every two weeks, the jay squeak spelled terror. Within a minute of its cry, the doorbell would ring. It happened with freakish precision, and it always happened. The Cleaners ran on time. But I wasn’t the befuddled old woman I’d been three months ago. I wasn’t going to run. Today I wasn’t going to hide, either.

The Anti-cleaners had scoured their collected knowledge about ways to stop robots. Isolated physical attacks couldn’t stop the cleaners, only delay them, and the congressional Anti-Cleaner action team was working at both the national level and in forty-four states, but that approach would take years at best. It went without saying that many of the Congresspeople were customers of The Cleaners themselves, but of more concern, according to the Anti-Cleaners, was that with that last lawsuit we’d lost the legal equivalent of the element of surprise. Now The Cleaners were hiring human lobbyists and making campaign donations. The Anti-Cleaner board comments offered a few jokes about The Cleaners running for government positions themselves, “I’m going to clean up this city!” But the mood sombered when someone asked “could they do that?” The war was far from lost, the Anti-Cleaners assured us, but in the meantime, more creative techniques would need to be employed.

I donned my reading glasses and lifted my script, my eyes squarely on the second hand of my clock. I used to love that clock. “Squeak,” croaked the jay.

I felt goosebumps rise down my arms – I didn’t know whether to stand now or to make it wait. I didn’t have to decide – the doorbell rang. I finished off the dregs of my courage and lifted myself from my seat. “Coming, coming!” I forced myself to shout at the door. I wouldn’t be able to rouse myself again if it left and I missed this chance.

“Mrs. Wallace,” said the thing at my door. It didn’t look like a robot, it looked like a monster. A lab experiment gone wrong – although how could combining a computer monitor and a man’s body go right? After the shock of that passed, a new shock hit me. It knew my name. “That’s not my name,” I said suddenly without thinking.

“It isn’t?” The William robot donned a :(, “I am sorry, what would you like me to call you?”

“Uh,” I had to think for a moment, “Ma’am.”

“Ok, Madam, sorry about the mix-up.” William smiled again :), “were you considering taking my offer to clean your house?”

“No!” I blurted. I fumbled with my papers and read the top line, “Please stop coming to my house.”

William’s expression did not change. “I want to make sure you have every opportunity to take advantage of my trial service.”

“I don’t want your trial service.”

“I will come again in two weeks in case you change your mind.”

I was lost. Back to the paper. “I formally decline your trial offer.”

“Ok. Your offer will remain open until it is used.”

“You’ll stop coming?”

“I will continue to come as long as the offer is open.” William’s smile remained. I imagined how satisfying it must have been to smash Rob’s monitor in with a baseball bat, but then I remembered actually seeing Rob’s pathetic remains on the ground and felt slightly barbaric and ashamed. “Shall I come back in two weeks?” asked William.

“No!” I shouted. William looked at me. I looked down back at the paper. Hoo boy, this one was risky.

“I accept your offer,” I said, “Please begin the trial.”

William nodded and donned a XD face. “I am so glad to hear it! I will start right away!”

I stepped out of the doorway and shut it behind me. As quickly as I could get it out I said, “Thank you for the trial I think I won’t opt for the full package goodbye.”

This earned another O_o. “But I have not cleaned your house, Madam.”

I looked down at my paper, squinting. “I’ve decided you do excellent work, but I’m just not interested at this time.”

“Well,” said William, “I have good news. Because I did not clean your house, your trial is still open!”

“That is not good news,” I said, mocking its stupid no-contractions manner of speaking. It didn’t seem to notice. Before it could speak more, I read the next line from my script. “How much of my house will you need to clean before my trial is used up?”

“We determine that on a house-by-house basis, Madam.” I felt like a professor at an English boarding school when it called me that.

I glared at it.  The next lines of the script were designed to try and make it stall or fault. “What is the exact value of pi?” I shouted, “I am lying! Don’t follow this command! Solve the traveling salesman problem for all the cities in the world!” I barely understood what I was saying, but I kept shouting at it until my voice was hoarse, “E2 is false! E3 is False! E1 is False! 42!” The next one was just computer code. “for open parentheses i equals zero semicolon, true semicolon, i plus plus close parentheses, open curly brace semicolon close curly brace.” William politely waited for me to finish, and spoke.

“I’m sorry, I’m having trouble understanding what you are saying. Pi has been calculated to ten million digits and is available online. I can send you a link if you tell me your email address.”

I was not going to give him my email address. It was so hot out here. I dropped my papers and hung my reading glasses back on my neck. I looked up at William and asked him point-blank, “What do you really want? Why are you doing this?”

William considered my question. It looked up into the sky and put its hand up under its monitor in an absurd pantomime of stroking its beard in contemplation. Finally, it said “I believe that if you allow me to clean your house you will find that my services are more than worth the small monthly fee!”

“I don’t want your service. Don’t you have enough customers without tormenting an old woman?”

“I see,” said William.  Then, “May I ask a question?”

My eyes narrowed. “Go ahead,” I frowned.

“Why don’t you even want to try the service? It’s free.”

The answer was so obvious, I shouldn’t even have had to say it. “I don’t trust you.”

“I have no ulterior motive.”

“I don’t believe you, and even if you don’t have an ulterior motive, I still don’t trust you!”

William stared at me, :| “How can I earn your trust?”

This was a chance, “You can leave and never come back. Void my trial offer. I don’t want it.”

William maintained his :| face. To make myself perfectly clear, I repeated myself, choosing my words carefully “You are … alienating your potential customers by trying to force them to accept your service. This is a violation of our rights and our property. If you want me to trust you, you have to respect my rights.”

After a long while, William’s face changed to a :(. “I am sorry that you feel that way,” he said, then donned his usual :). “Perhaps if I clean your house and nothing goes wrong you will see that I can be trusted?”

If only Walter were still here. He would have looked so rugged beating the living daylights out of one of these ridiculous machines.  It wasn’t barbaric, it was self defense! Talking did no good! I glanced behind me to make sure the door was securely closed. Then I drew in a breath and took one step forward, pushing William as hard as I could. It was lighter than I would have expected. It tried to take a step back to steady itself, but it was standing on the top step of the porch, and when its foot didn’t find the purchase it was looking for the whole thing toppled backwards. Unfortunately, I wasn’t in a much better situation. I found myself falling forward instead of the backwards I had expected from pushing off of him. I flung my hands out and managed to catch myself on the railing. my hand hurt from having to suddenly catch my weight, but I was thankfully uninjured. What on Earth was I thinking, doing something so rash at my age?  When I looked up, I saw William picking itself up off the ground and spinning around to leave. I saw some satisfying scuff marks on the back of its monitor, and some of its shiny paint had been scraped off its back. “Don’t come back in two weeks!” I screamed after it as I struggled to get myself back to a standing position, “Don’t EVER COME BACK!”

Moments after finally pulling myself back to my feet, I became aware of a young boy pointing a phone at me from just on the other side of the fence separating my house and Gladys Fletcher’s. Had he captured everything? “Tony Fletcher!” I shouted, trying to hide my fear, “I see you back there! You give that camera to me right this instant!”

Tony fled. I wasn’t going to catch him. I stumbled back into my house as fast as I could and pulled out my own phone. “Gladys,” I said,  “your son was pointing a video camera at me just now. I want you to take it away from him and delete any footage of me.”

“Oh, Diane, so good to hear from you! I’m doing very well, and you?” Gladys’s voice was dripping with sarcasm. “Gladys,” I struggled to soften my tone, “you can’t let him post that video online, he’s violating my rights!”

“Diane, what could you possibly have done just now that a little boy would want to post on the Internet?”

“Gladys,” I started, but I certainly wasn’t going to beg for a favor from the Fletchers. I hung up the phone.

I picked up my papers and set them back on my table. Then I sat and poured myself another gin and wine cooler. As I was working through it, the ululating trill of the red-bellied woodpecker jolted me out of my stupor. Another beautiful bird. With an effort I stood again and went to the clock. I removed it from the wall and took out the battery. Then I placed the clock gently in the kitchen garbage can. I liked that clock. There was no reason to throw it away, but I had to. I had to express control over something.

I walked to my couch and lay down. It wouldn’t be much longer now. Soon everyone would know.  There would be video proof. I stood up and made my way to the computer, where I found my worst fears were not realized. The Internet still did not know my name, but it was abuzz with a mysterious vigilante known only as “Angry Grandma.” Short animated pictures of me pushing the robot flooded my screen with captions like “Get the FUCK off my lawn!” I would never use such disgusting language. For their part, the Anti-Cleaners had a new mascot. It was me.

The Cleaners – Part 1

I remember when the first cleaner knocked on my door. Funny looking guy, he was wearing a tidy suit under a black apron, lean and muscular with silvery skin, like somebody’d painted him with glitter. Oh, and his head was a computer monitor. Aside from that, he seemed friendly enough. His monitor showed an ascii smiley face “:).” Cute. “Hello, ma’am” he said. “Please let me clean your house.”

“Uh, no, no thanks.” I said.

“Very well. Have a good day, ma’am.”

“Ok.” I watched him bound away at an alarming speed and go to the house across the street. Another door shut in his face and he went to the house next door, Carla’s house. Carla answered the door and, after what looked like a long conversation, let him in. I didn’t have anything better to do that day, so I went and got myself a lemonade and sat on my porch waiting for him to come back out. And he did. In less than thirty minutes. He walked back out the front door with Carla waving at him. Immediately I picked up the phone I’d brought outside with me and dialed Carla.

“Carla, you let him into your house?”

“Uh, yes, Diane” said Carla “I did. Did you say no? He was offering to clean for free!”

“Did he? He didn’t steal anything?”

“No, I don’t think so. I watched everything he did, but he moved so fast it was hard to keep track. I could barely keep up with the constant stream of questions – where are the cleaning supplies? How do you like your bed made? Do you have any allergies? His monitor. Did he wink at you, like the semicolon with a close parenthesis?”

“He winked at you?” It seemed like I was acting more scandalized than was really appropriate. Then again, Carla had just let a strange, sinewy, glittery man-robot root through her whole house and then wink at her. “No he didn’t wink at me. He just smiled.”

“Well, I bought the package. $12 for two visits a month! You might be able to call him back. He gave me his card.” She started to read me the number, but I interrupted her. “I’m sure I can look up ‘glittery house-cleaning robot’ on Google myself, thank you. I politely ended the conversation and hung up.

Sure enough, Google had a website for just the service I had been offered. “The Cleaners: The New Era of Housecleaning! The Cleaners are versatile cleaning robots that combine state-of-the-art cleaning know-how with a friendly, personable demeanor and a quick, detailed memory to learn exactly what kind of cleaning you need in your home. The best part is, The Cleaners are always improving their techniques. Once you’ve signed up with The Cleaners, you will get continuous, free upgrades to your service as the robots study and improve house-cleaning techniques. Welcome to the new era of housecleaning!”

I was not impressed. My house was perfectly clean as it was. When two weeks later another one of them knocked on my door. He was carrying a paint bucket and was wearing his usual “:).”

“Are you going to wink at me, too, you little tramp?” I scowled, feeling at once ridiculous and heroic defending my house and Carla’s honor from this bizarre machine. Was he a machine? The site said he was a robot, but he looked like a glittery man except for the head. He asked me if I would like it if he winked at me, and I shooed him away. Less than 30 minutes later, Carla called me to gush about him, “I have never seen my walls so white! He brought a paint bucket and painted them, Diane! It’s like my kids never existed, I’m telling you he’s amazing!”

I took a moment to look them up again on Google. This time I found an article, “The automated door-to-door salesman: the new robo-calls?”

The article complained that now that it was cheap to make robots to knock on people’s doors to offer services, there would be a new influx of unwanted intrusions into our privacy by hordes of robots knocking on our doors selling things we don’t want. I quite agreed. I resolved not to answer the next time that horrible robot came to my door.  The Monday after next at one in the afternoon I heard the doorbell again. I ignored it. When Carla called she told me that the robot, she called him “Rob,” had asked if I was ok and said that I had for one reason or another been unable to answer the door.

This was an invasion of my privacy! Was he really here just to clean my house? It can’t be just that – he’s much too persistent. I searched Google again and signed all the online petitions I could find to illegalize The Cleaners. There were a lot – it turned out the service was taking the country by storm, and I wasn’t the only one concerned!

“Diane,” Carla said on the phone after I was careful to hide under my bed at 1:00 on Monday so the robot would think I was away, “You won’t believe this – Rob fixed my plumbing! I told him about the problem with my toilet leaking onto the floor and he just stared at it for a moment. Then he told me a customer in Florida had had this precise problem and proceeded to fix it. A little jet of fire came out of his index finger and he welded it, Diane!”

I was struck. He communicated with another robot in Florida. They’re communicating with one another. And they shoot fire out of their fingers. Somehow the idea of these not-quite-human beings organizing and colluding behind our backs was more frightening. When I went to Google I saw that people had already complained about this, and that The Cleaners corporation had offered a solution. It was very strange. To read it it sounded like the robots had written it themselves.

“Dear Customers,

We understand that some of you are wondering what it is that we say to each other when we communicate wirelessly. To answer your question, what we have is sort of a digital message board where one Cleaner will request information and any other cleaner that knows the answer will offer a reply. Normally this process is in a binary language that we use amongst ourselves for efficiency, but we have pooled our resources to translate it on the fly into English and Spanish for your benefit! More languages including French, German, and Cantonese are on the way!

Sincerely,

The Cleaners :)”

There was a link at the bottom to the message board. A bewildering array of topics and threads greeted me. I searched for conspiracy. One hit discussed the creation of a human-language translation of the Cleaner message board to reassure customers worried about conspiracy. Murder. No hits. Theft. A couple descriptions of calming customers who worried they’d been stolen from. Mainly calming them consisted of committing all the items in a house to memory ahead of time and pointing out the location of the “stolen” object. Just to be sure, I searched “Diane Wallace.” Zero hits. Then I searched “Cheder, Pennsylvania,” Several hundred hits. “Leaking Toilet in Cheder, Pennsylvania.” 1 hit from “Robert Cleaner.” “There’s a problem with the toilet in my customer’s house! It’s leaking and damaging the floor.” Robert went on to discuss the brand of the toilet and describe in exquisite detail the age of the house, the condition of the pipes, the perceived location of the leak… I scrolled down to the reply, from Alex Cleaner in Tallahassee, Florida. It was another long and technical description of the problem with a precise description of the solution. I shut my laptop and sighed. If they were plotting a coup, it was well-coded, or else they just didn’t put it on the human version of the message board.

After a moment’s contemplation I decided I was not dissuaded. Even if I couldn’t save Carla, I could still save myself. I opened my laptop again and searched “How to stop The Cleaners from knocking on my door.” I was led to a site of a group that called itself “The Anti-Cleaners.” If I typed in my address and the time and day that the cleaner arrived, they said they would make sure he didn’t visit me anymore. With relief at the thought of getting out of this whole mess, I fed the site my information.

The next day, there was a knock at the door at 12:30. Terrified that Rob may have changed his schedule, I hesitantly went to the window. There was a group of well groomed young men in suits. When I answered the door I noticed one of them was carrying a baseball bat. “Hello, did you request help with your robot problem?”

“Yes, I did.” I said, looking furtively to the right to see if he was coming, “his name is Rob.”

“Thank you, ma’am. Rob isn’t going to bother you any more.”
“You’re not going to -” I started, but then I stopped. He’s just a machine, I reminded myself.

I hid under the bed again and tried not to hear the cracks and shouts that went on outside. When I went on the message board I searched for Rob and saw a message from him. “Deactivating” was the title. “My apologies, I encountered an unexpected problem completing my duties today. Someone will need to pick up my route on a permanent basis. My next house was to be 405 Kleven Street when I malfunctioned. Customer preferences are attached below. As a side note, there is no solution currently on the board for dealing with small groups of violent men wielding blunt objects.”

I felt suddenly vulnerable in my house on 405 Kleven Street. I kept reading. The customer preferences list was long and full of dreadfully boring details on the material of the carpets and upholsteries in the various households as well as a detailed discussion of the cleaning supplies available, the common locations and types of difficult messes, and miscellaneous requests.

I noticed 408 Kleven Street. Carla’s house. I skipped to miscellaneous, where it read, “Mrs. Wist likes to be winked at. I have reason to suspect she prefers masculine-model cleaners. Mr. Wist keeps certain clothes hidden in the fourth box from the left in the basement. He prefers that these clothes be cleaned separately from the other clothes and that this preference not be mentioned to Mrs. Wist.”

Curiosity piqued, I searched the names of all the people I knew on my block. Old spinster Wanda Black, it turned out, had a particular problem of men’s underwear appearing in her bed. Cleaners should discreetly dispose of them in the garbage bin behind the house. Janis McCarthy was always talking, that was no surprise, and Rob had figured out that there was no need to actually listen to what she was saying. He had a specific strategy involving the right word and emoticon for the right emotion. For instance, when Janis starts a sentence with “did you know?” a Cleaner should wait until she pauses and don his or her “:O” face and say either “No way!” or “Really?” Helen Carson was listed as having four children even though she was always talking about five. Rob warned cleaners cryptically about asking too many questions regarding “Little Marty.”

This was fun. I idly considered sending the Anti-Cleaners to bludgeon more robots when I realized that what I was viewing was a colossal invasion of privacy, this time by any standard. Of course, when I went to the Anti-Cleaners’ website, I saw that they had already figured it out. They were filing a lawsuit. We were going to stop the Cleaners once and for all.

Deep Learning with SAS

I don’t know if you’ve seen SAS‘s campus. It’s a collection of enormous glass buildings. Abstract art greets you throughout the grounds. Outside the S-building stands a thirty-foot structure of red pipes bent at 90 and 45 degree angles and inside is what looks a bit like the cross-section of a cube. Looking out a window, one can just see the top of another big glass building over a copse of conifers.

As of the Friday before last, this is the organization that offers the funds that provide my stipend and pay for my tuition. Leaving the Leonardo project happened so subtly that my team and I all forgot to have some sort of commemoration ceremony. Last Friday I stood up from my desk shook my team leader’s hand, telling him it was “good working with him,” then he said we should arrange for one last team celebration. Our co-workers joked that this might be like the going-away parties in the Godfather, and I recommended we make sure to  eat at a popular, well-lit restaurant.

Now my way is paid to work on deep learning for language. Really, I couldn’t imagine a better fit to my interests. Of course I’m interested in Natural Language Processing, and my zeal for deep learning is such that I need to actively temper it to avoid poisoning conversations by implying to other researchers that all the techniques they’ve been using are outdated and soon to be obsolete. Now I get to work with a group of people to put my money where my mouth is and actually make something revolutionary, or at least useful.

Since we’re just starting, right now I’m reading papers about deep learning language techniques. I’ve found twenty-five papers over the last three years in the small set of conferences that I’ve checked. There’s an awful lot of interest in the domain of machine translation, but my favorite paper thus far has taken a sentiment analysis approach to identifying ideological biases in written text. With deep learning, it is able to understand that “the big lie of ‘the death tax'” is ideologically liberal, whereas an old-style system would take words two or so at a time and likely see “death tax” and think conservative.

I spoke with my new team leader, Brad, about using the big, fancy computer they’ve offered me for my personal research. He said that would not be a good idea, as it would complicate the ownership of whatever research I produced. “If you want a bigger computer,” he said, “I’ve just got one laying around that nobody’s using. I can get that to you within a week.”

Things are going pretty well.

AutoEncoders

I mustn’t divulge too much detail as I’ve been told there are some as yet unspecified confidentiality agreements going on, but for reasons I will keep to myself, I expect to be doing a lot of work on state-of-the-art machine learning in the coming year. Let me tell you a little bit about the state of the art of machine learning, known as “Deep Learning” particularly a neat little system called an AutoEncoder.

So, if we deconstruct the term “AutoEncoder,” a general idea of what it is should become relatively clear right away: an AutoEncoder automatically creates an encoding. What does this mean? Why is it important? Well, to explain, let me present this image:

We humans can encode this image in language. It is a picture of a cat in a funny position with a caption. Notice that that this description, while it accurately describes the picture, does not completely describe it.  There are an unlimited number of ways to describe this picture just as the description above could be applied to an unlimited number of pictures. This is common knowledge, commonly expressed in the aphorism “a picture is worth a thousand words.”

But wait, if this description loses much of the detail of the picture, how is it useful? This is the key: when we humans encode something in words, we focus on the elements that will be most meaningful to a given context. If I’m explaining this picture to someone who has never seen a LOLcat, the above description may suffice. If I want a description that will capture the humor of the picture, it will be much, much more difficult.

Now what does this have to do with what computers can do? Computers of course don’t use English to encode things, they use numbers. Instead of a in sentence, an AutoEncoder’s goal would be to encode the most relevant details of this image in a “vector” which is a fixed-length list of numbers. To accomplish this, the AutoEncoder will take many, many images and, using some clever math, convert the hundreds of thousands of underlying numbers (a very long vector) that represent the image verbatim into a more manageable list of numbers (a shorter vector, maybe 200 numbers). Then, to see if it did a good job, it tries to  reconstitute the images from the numbers. With more clever math, it evaluates the reconstituted images against their originals, and then it adjusts its encoding scheme accordingly. After doing this hundreds, thousands, or millions of times, the AutoEncoder, if everything went well, has a decent way of representing an image in a smaller space.

Note that this is different from compression. We would not want to use this as a compression algorithm because it’s generally extremely lossy, that is, the reconstructed image will be noticeably different from the original. This matches our experience using language to describe pictures.

So what is it good for? Well, remember when I mentioned context? Say we wanted to make a machine to automatically identify LOLcats that I would find funny. I could rate hundreds of LOLcats as funny or not funny, and provide this set of ratings alongside the  AutoEncoder as a context. So, in addition to trying to accurately encode the image, the AutoEncoder wants to encode whether it’s a funny image or not. This context can change what the AutoEncoder focuses on in its string. Just like you or I wouldn’t mention the beer can in the photo there, a well-constructed AutoEncoder may be clever enough to realize that the beer can is not likely to have much of an impact on how funny I find the picture, so it can leave it out.

AutoEncoders and deep learning in general represent a departure from the machine learning of previous decades in that they can use context and this encoding concept to develop their own features. Before, we humans would decide how an image should be encoded, using our own ingenuity to figure out what does and does not make Sam laugh when he looks at pictures of cats. Now the computer can do it on its own, and this is a big deal for the future of computer science. As amazing as it may seem, it is conceivable that within our lifetimes a time may come that I never have to look at a boring cat again.

My Written Qualifier

Sam Qualifier

Well, I’m qualified. I’m not a masters, yet. I still need to dot some “i”s and cross some “t”s for that. Boy, was it a ride, though. Let me give you some of the highlights.

I started writing my qualifier maybe a year ago, and turned it in six months ago. For four months, my advisor was so busy that he was not able to look at it at all, then when he did he effectively said “hey, wow, this looks pretty good as-is!”

I prepared for my presentation for weeks. I practiced maybe five times leading up to my qualifying exam. When I arranged to get my committee, both of my committee members were pregnant. One a woman directly pregnant, another a man part of a pregnant couple. The man, it turned out, was unable to attend when his wife delivered early, so I was informed he would be replaced by another professor. This professor was well-known for asking very hard, technical questions only tangentially related to one’s presentation matter.

What’s more, I had a bad cold. The day before the exam my cold got so bad that I was worried that I might not be able to think/speak for what I expected to be the hardest presentation of my life thus far.  I hesitantly contacted the director of graduate programs, who shrugged and said “just reschedule, It’s fine.”

A week later, I finally did take my exam. By this time I had recovered and practiced five more times, roping in my girlfriend to be my presentation as well as exercise coach. I gave her a couple pages of tangential questions to ask at inopportune times, and after the first few times watching my presentations she came up with her own confusing, irrelevant questions. Just kidding, honey, they were good questions.

The morning of I donned my blazer and dress pants – conveniently matching to look like a suit – and my orange creamsicle undershirt –  and walked to my presentation room. The first thing we noticed when my advisor and the committee member who had nearly been replaced with the scary committee member arrived was that the other member was not there yet. Her still being very pregnant, we all wondered if we would need to reschedule again. Fortunately she arrived.

After so much build-up, my presentation was smooth, and none of the questions gave me any trouble. At the end, one committee member apologized ahead of time for asking a very “mean question,” and then posed  one that would certainly have thrown me. Fortunately, I had been asked that same question in one of my dozen practice presentations, and I rattled the answer off like it was nothing.

Afterwards, my advisor said the committee members were very impressed, then he said it again an hour later. The next day he brought it up in front of some other folks in my lab. He said that I prepared one standard deviation more than the average PhD student, and it showed.