It is painful, and nearly useless to know. But once you understand it, you now know how a computer “works”. It is impossible to understand the hardware/software interface without an knowledge of assembly. In particular, if you are in the computer security field, you have to understand how things really work. These days, I’ve developed a hobby of reverse engineering things and hacking around at the binary level. If you are a high level programmer (python, ruby), odds are high that you haven’t had to worry about memory management and pointers. If you have a c, FORTRAN or (mid-level) background, odds are high you haven’t had to worry about the stack or the different op-codes for system calls and interrupts, let alone which internal register things are stored in. However, these interactions are the exact thing you have to understand to “de-blackboxify” any computer. This makes a trip through assembly a necessary stop for everyone doing research in the security field.
The first thing for me is to get started with the most basic program possible. Since I’m just like everyone else in computer science these days, I use 64 bit OS X locally which is a flavor of the BSD operating system so I’m going to use nasm on my mac (brew install nasm) to assemble my code into an object file. Normally a compiler like gcc (or clang, etc) will turn c into assembly, and then object code, which then, in turn, is turned into the machine code specific to your system and processor. Think of object code as a numeric version of the assembly. It is an excellent exercise to translate assembly into opcodes, and is extremely complicated with lots of binary math to get the instruction sets right. If you want to play around with this, I recommend you check out this interactive page which converts the asm opcodes for you. I’ve met two people in my life who can actually code directly in opcodes, and once you nug through it once, it is mind-blowing that folks can do that much math in their heads.
The first command, nasm -f macho64 hello.asm, turns the code above into an object file. An object file is a file containing general machine code which are generally called opcodes. These opcodes are relocatable and usually not directly executable. There are various formats for object files, and the same object code can be packaged in different object files. It’s mostly machine code, but has info that allows a linker to see what symbols are in it as well as symbols it requires in order to work. (For reference, “symbols” are basically names of global objects, functions, etc.)
Square roots for perfect squares with integer results
This works fine, now we want to make the world’s most simple square root function. In order to think things through, I wrote up a basic (and verbose) function in c to find the square root of 64, or of any number that is a perfect square with integer results.
#include
int main()
{
int len;
int num;
int sqrt;
int lead_bytes;
num = 64;
lead_bytes = __builtin_clz(num);
len = (32-lead_bytes);
len = len/2;
sqrt = num >> len;
printf("{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}d",sqrt);
}
With this as a guide, I then did the laborious process of translating this into x86 64 assembly for the mac architecture:
Woot! In my mind I lose some leet points for using the default c-library, but it is still the most low-level hand-crafted code I’ve put together. For assembly, I’ve found that documentation is critical. Every implementation of every architecture is very specific. For example, mac requires _printf not printf which windows would require.
Now, if we want to make this interactive I’m going to make command line arguments possible, in addition to creating an external c library and linking to this.
Getting the files to run with command line arguments
Here knowledge of C translates over nicely. In C, main is just a plain old function, and it has a couple parameters of its own:
int main(int argc, char** argv)
If you follow the docs, you see that argc will end up in rdi, and argv (a pointer) will end up in rsi. Here is a program that uses this fact to simply echo the commandline arguments to a program, one per line:
EDIT: I found a more robust way that works.
Up next, running as a c-shared library (to run in Julia)
Recently, I had to produce form 40A’s in order to claim military leave. As a computer scientist, I found the process to be horrific and felt compelled to share my solution. I didn’t get much help with UTAPSWEB FOR IMAs CHEAT SHEET VERSION 101210. That document is filled with unhelpful and unprofessional statements like:
Changes made to Form 40As because you were just ‘trying something’ may not be able to be resolved.
Oh, well, I Am Alone once again. I’m a hacker, famous for ‘trying something’, well here goes . . .
Should you ever find yourself in this situation, navigate to “Automated Form 40s”:
From here, I had to look at the source code to decode the buttons. The icons kinda make sense, but they are about 10px high and don’t even have mouseover text. Ok, the source can’t lie:
Based on this, I clearly wanted the
MainContent_imgPrintMassSelect
. After clicking this and waiting for the long update to happen. You can select multiple 40As:
So from here, the
imgPrintSelected
button looked like the next logical step. Make sure to allow popups (nice 1990s web-practices!). The result was an unexpected:
So that doesn’t work. Let’s look at the traffic and dive into the form. First, there is no form. They are counting on a click event with js in the div header.
So the important code is clearly in there, there are single click events, double-clicks and even an attempted contextmenu override.
At this point, I’m getting in too deep and just need to get 16 IDTs printed. Clicking on one at a time will take 20 minutes, so I wrote an adobe script to automatically change the dates. Did anyone else have better luck on this? I’m considering writing a new front end to all of this that scrapes the back-end. Any other developers want to help me with that?
It is not easy to quickly decipher military pay tables for us IMAs. In order to do some recent financial planning, I had to calculate my pay. I’m an O-4 with between 14 and 16 years of service. Here is what I did.
IDTs
You can find the official military pay tables from the DoD comptroller but I found militaryrates.com to have the 2015 data that I couldn’t find on the official site.
Here, I saw that based on my years of service, my drill pay is \$962.83, which is 4 IDTs, or 16 hours, so I get paid \$60.17 an hour. For 48 IDT’s (Cat A), this means I get paid
for IDTs. Drill pay is higher than basic pay. I assume this is because drill pay is burdened by all the things you don’t get as an IMA: health benefits, BAH, BAS.
Annual Tour
Now, to calculate the annual tour (AT) we use the regular military pay tables. On the first page of the scale is your monthly military pay. First, find the pay that applies to you based on your rank and time in service. If you divide that number by 30, that gives you your daily pay. Multiply that number by the number of annual tour days you are required to do (14 in my case as a reservist) and you’ll have your before-tax annual tour pay.
then $ \$ 240.71 \times 14 = \$ 3,369.90 $, which is appears to be exactly half of what I would get if I got IDT pay for the annual tour.
All together, this means \$15,000 a year in gross income from the reserves.
How do you value the retirement benefit?
To collect on retirement benefits, you have to attain age 60, not be entitled to receive military retired pay through any other provision of law and complete at least 20 years of qualifying uniformed service. So how much would I have to invest this year to have this benefit?
Should I make it to that age, on Tuesday, August 12, 2036, I will be 60 years old (21 years from now). Here I have to make some assumptions:
I retire as an O-6 in 6 years from now.
Officer pay rises with inflation
Discount rate of 6{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}
So, 6 years from now O-6’s will be making a base salary of \$119,726.16. The defined benefit plan for DoD is 50{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} highest salary or roughly \$60,000. In then-year dollars that would be \$71,479.65 a year. So, avoiding all fees, if I wanted to have enough cash to provide me with an annuity that paid \$71,479.65 a year in 2036, I would have to have \$1,191,327.50. So, if I wanted \$1,191,327.50 in 2036, how much would I have to save per year when I started the reserves? It is easy enough to compute the payment amount for a loan based on an interest rate and a constant payment schedule. In my case, this comes to \$20,138.61 a year that I would have to invest to get that benefit. You could see the math behind all this on wikipedia. Now, one might question the value of \$71,000 in 2036. If we experience several years of high inflation (which we will) that might not be worth much. For example, in current year dollars assuming a 4{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} rate of inflation, the retirement benefit is only worth roughly \$31 thousand annually.
What about other benefits?
Now, you also have to compute the value of medical benefits, etc. Military discounts, commissary, etc, which are going to be highly dependent on the individual, but I would personally pay no more this year than $500 to get. (The medical benefits might be huge as might the post-9/11 GI bill.)
The other big benefit is career diversity and having a broader network and official connectivity to two government organizations. This alone might be the biggest benefit of the reserves if a member is very transparent and wise in how they use this opportunity.
So, in total, I would say that I make \$35,500/year in reserve benefits. What is the downside? I could be spending my reserve time on my main career which could lead to more salary in the right field. I could also be building a start-up with that time that also might pay off and doing something that might be closer to my passion. I could be investing in my faith, house, family or health. However, the fact I work for the government means that I can actually do a form of approved side work. Other jobs/consulting would be much more difficult and uncomfortable. I could certainly have much less stress if I gave this up.
Would love any thoughts, particularly those which correct errors in my thinking above.
One of the more frustrating things about the reserves: obscure codes like those above. I find it incredible how poor reserve support is, and unnecessary obscurity shouldn’t be tolerated. For those who wonder what these codes actually mean:
Provenance is the ability to record the history of data and its place of origin. In general, it is the ability to determine the chronology of the ownership, custody or location of any object. The primary purpose of tracing the provenance of an object or entity is often to provide contextual and circumstantial evidence for its original production or discovery, by establishing, as far as practicable, its later history, especially the sequences of its formal ownership, custody, and places of storage. While originally limited to determining the heritage of works of art, the term now applies to wide range of fields, including archaeology, paleontology, archives, manuscripts, printed books, and science and computing. The latter is the context most relevant to my field of computer security.
In the context of data provenance, provenance documents the inputs, entities, systems, and processes that influence data of interest, in effect providing a historical record of the data and its origins. The generated evidence supports essential forensic activities such as data-dependency analysis, error/compromise detection and recovery, and auditing and compliance analysis, including the ability to detect advanced/persistent threats. Data provenance can provide a full historical record of data and its origins and the provenance of data which is generated by complex transformations such as workflows is of considerable value to scientists. From it, one can ascertain the quality of the data based on its ancestral data and derivations, track back sources of errors, allow automated re-enactment of derivations to update data, and provide attribution of data sources. Provenance is also essential to the business domain where it can be used to drill down to the source of data in a data warehouse, track the creation of intellectual property, and provide an audit trail for regulatory purposes.
The use of data provenance is proposed in distributed systems to trace records through a dataflow, replay the dataflow on a subset of its original inputs and debug data flows. In order to do so, one needs to keep track of the set of inputs to each operator, which were used to derive each of its outputs.
The w3c defines provenance as the ability to record a resource in order to describes entities and processes involved in producing and delivering or otherwise influencing that resource. Provenance provides a critical foundation for assessing authenticity, enabling trust, and allowing reproducibility. Provenance assertions are a form of contextual metadata and can themselves become important records with their own provenance.
Why do we care?
Because provenance provides a critical foundation for assessing authenticity, enabling trust, and allowing reproducibility and assertions of provenance can themselves become important records with their own provenance. The widespread use of workflow flow tools for processing scientific data facilitate for capturing provenance information. The workflow process describes all the steps involved in producing a given data set and, hence captures it provenance information. Provenance can be used to record metrics such as data creator/data publisher, data creation date, data modifier & modification date, or data description.
There are two major strands of provenance for computer science: Data Provenance and Workflow Provenance. Data provenance is fine-grain and is used to determine the integrity of data flows. It is a description of the origin of a piece of data and process by which it arrives in a database. By contrast workflow provenance is coarser in grain. It refers to records of history of the derivation of the final output of workflow and is typically used for complex processing tasks. Fine-grain provenance can further categorized into: where, how and why-Provenance. A query execution simply copy data elements from some source to some target database and where-provenance identifies these source elements where the data in the target is copied from. Why-provenance provides justification for the data elements appearing in the output and how-provenance describes some parts of the input influenced certain parts of the output.
Hello fellow IMA. My apologies to you. Life is not easy. In the civilian world, you work hard, play office politics and with a little luck you might get promoted. Not so in the reserves. Here your promotion depends heavily on your ability to decode a bunch of Air Force personnel jargon and to make a lot of non-cooperative admin types take care of someone who they really don’t see as their responsibility. I hope my story helps you out.
To start preparing for a recent board, I had to look up some basic information to answer the following:
When is my board?
How do I know if I’m eligible?
When is my PRF due? When does it have to be signed and where does it need to be delivered to?
How do I review (and potentially change my records)?
PRF
Before answering these questions I had to write my PRF. Why do IMAs write every word of their PRFs and OPRs? Because IMAs are always shafting their reserve boss because of the demands of our main job and the last thing we want to do is have someone go through the torture of the AF evaluation system when we’ve been so lame.
But nothing is easy — the only time I have to work this is while I’m flying from DC to Vegas and I’m on my Mac at 35 kft. I have a draft of last years PRF but it is in $xfdl$ format. My mac is not any mac, it is a government mac from my day job so I can’t install any software. Oh yes, this is totally doable, I’m an engineer. Bring it. So the XFDL is base64 zipped. To learn this, I connected to a free cloud based bash shell VPS (seriously cloud 9 IDE for the win) and cat the top of the xfdl and see:
so no probs here . . . because I’m on a shell with root I can use uudeview under linux to decode a xfdl into a zipped xml file and then extracted it to view in emacs. Happy to explain this in more detail if you email me at tim@theboohers.org for other questions, I recommend you call the total force service center at Comm 210-565-0102.
What do non-hacker IMAs do? Ok so I can parse XML easily enough to get the following from here.
The document to make sure you have in your hip pocket is AFI 36-2406 OFFICER AND ENLISTED EVALUATION SYSTEMS. It is probably the worst written document possible for quickly finding what you need, but it is the guide for how this is all supposed to work.
The most helpful document was the ARPCM_16-02 CY16 USAFR Lt Col Convening Notice, which I dug around on MyPers to get. From this document I found out that I would need a date of rank for a Lieutenant Colonel Mandatory Participating Reserve (PR) board to be less than 30 Sep 10. I can see that my DOR is 29 APR 2010 and that fits in the window of the oldest and youngest members for the board:
DAILEY, MELISSA A./30 Sep 10 VANMETER, BRETT A./1 May 02
When is my PRF due? When does it have to be signed and where does it need to be delivered to?
From 36-2406, I know then that an eligible officer’s senior rater completes the PRF no earlier than 60 days prior to the CSB: which for me is Thursday, April 14, 2016.
From the table above, I see this confirmed that my senior rater (the USD(P)) has to sign the document between 14 Apr 16 and 29 Apr 16 and I get the completed document by 14 May 16. I can’t find how the PRF gets to the board, but I’m just going to bug the unit admin until I can confirm the document is in.
How do I review (and potentially change my records)?
Check your records on PRDA. So I was missing two OPRs and an MSM. Wow. The key here was working my network and finding the (amazing) admin at ARPC/DPT who had direct access to the records database and was able to update it for me before the board.
While I can’t get into the background here, I was recently asked to describe the most geeky event in history. Something like this is hard to bound and as I asked those around me I got lots of answers from moment the zero was invented (“without that, you got nothing”) to the first demo of the transistor (pretty good one) to the Trinity Test for the Manhattan Project. All of these were interesting, but “Geeky” implies not necessarily profound, but technical, funny, entertaining and weird. As I tried to put together my answers, I came up with the following goals:
geeky stuff is funny, so the moment should at least make you smile
geeky stuff is esoteric and should surprise and be a little weird to the general public
geeky stuff is technical; a geeky moment should be the culmination of a long technical slog and unveil a new tech in a novel way
geeky stuff is changing the world, so this moment should have at least a brush with history
So, I came up with four events: (1) A profound historical event, (2) A funny event, (3) a profound economic event, and (4) an tech awe-inspiring event.
The Profound Event
On 9 December 1932, 27 year old Marian Rejewski (a Polish mathematician) was given some German manuals and monthly keys for their Enigma. On top of his recently developed theory of permutations and groups, this enabled him to work out the Enigma scrambler wiring after he had the insight that the Nazi keyboard was wired to the entry drum, not in keyboard order, but in alphabetical order. He describes what happened next: “The very first trial yielded a positive result. From my pencil, as if by magic, begin to issue numbers designating the wiring”. By Christmas he was decoding nearly all German communications. Not only was this one of the greatest breakthrough cryptologic history, but it eventually resulted in the British capability listen to every conversation made between Germans. This played a big role in decimating a superior German Navy and Luftwaffe, and possibly swaying the war in favor of allies, in addition to starting the modern field of computer science.
The Funny Event
In the fall of 1971, the twenty-year-old Wozniak was inspired by Esquire to build a “Blue Box” that enabled him to create the 2600Hz tone to control the, then analog, phone network. He took the name “Berkeley Blue” while his 17 year old business partner Steve Jobs took the call sign “Oaf Tobar.” After a visit to the original phone phreak, Captain Crunch, and a short detainment by the Police, he decided to put his blue box to good use and do something both epic and geeky: crank call the Pope. Using his blue box he managed to route his call to the Vatican. “In this heavy accent I announced that I was Henry Kissinger calling on behalf of President Nixon. I said, ‘Ve are at de summit meeting in Moscow, and we need to talk to de pope.'” The Vatican responded that the pope was sleeping but that they would send someone to wake him. Woz arranged to call back in an hour.
Woz recalls, “Well, an hour later I called back and she said, ‘Okay, we will put the bishop on, who will be the translator.’ So I told him, still in that heavy accent, ‘Dees is Mr. Kissinger.’ And he said, ‘Listen, I just spoke to Mr. Kissinger an hour ago.’ You see, they had checked out my story and had called the real Kissinger in Moscow.”
Aside from the raw audacity required to prank world leaders, there is something very geeky about building a piece of hardware out of spare parts, mastering the world-wide phone system and roping in a key source of the modern desktop, desktop publishing, digital entertainment and mobile devices in the operation.
A Profound Geeky Economic Event
The moment when Dan Bricklin – wrote the original spreadsheet program, VisiCalc, on an Apple II. It is hard to overstate the impact spreadsheets have had on the modern economy and the PC industry . . . and it was written in assembler, which adds major geek cred.
Tech Awe-Inspiring Event
I’ve always been in awe of Von Neumann and his impact on science and defense technology. In particular, in May 1945, he hopped on a train from Aberdeen, Maryland to Los Alamos, New Mexico. At the time, he was solving some of the hardest problems on the Manhattan project and was traveling to go provide technical advise leading up to the trinity test. He had already established game theory, numerical analysis and a good bit of quantum physics. As a side project, he had just seen a demonstration of the ENIAC computer and had been asked to write a memorandum describing the project. I’ve written a lot of memos describing government projects, but von Neumann’s 101 page hand-written “The First Draft of a Report on the EDVAC” was finished by the time he arrived in Los Alamos. He mailed it to his colleague Herman Goldstine, who typed it up, got it reviewed, and published it the next month (without any patent). His memo described the architecture of the EDVAC, the follow-on to ENIAC which was to become the world’s first von Neumann machine and was the first published description of the logical design of a computer using the stored-program concept that formed the blue-prints for 50 years of computers.
You can’t work long in computer science until you have to understand the halting problem. The halting problem is concerned with determining whether a program will finish running or continue to run forever. It has a storied history. In 1936, Alan Turing proved that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist. His proof was made famous since in this paper, he provided a mathematical definition of a computer and program, which became known as a Turing machine and the halting problem is undecidable over Turing machines and is one of the first examples of a decision problem.
Programmers generally try to avoid infinite loops—they want every subroutine to finish (halt). In particular, in hard real-time computing, programmers attempt to write subroutines that are not only guaranteed to finish (halt), but are guaranteed to finish before the given deadline.
But isn’t it easy to avoid infinite loops? Why is the halting problem so important? Because a lot of really practical problems are the halting problem in disguise. For example, say you want a compiler that finds the fastest possible machine code for a given program. You have JavaScript, with some variables at a high security levels, and some at a low security level. You want to make sure that an attacker can’t get at the high security information. This is also just the halting problem. Or, for example, if you have a parser for your programming language. You change it, but you want to make sure it still parses all the programs it used to. This is one more example of the halting problem. If you have an anti-virus program, and you want to see if it ever executes a malicious instruction, this is another example of the halting problem in action.
Moreover, the Halting problem lets us reason about the relative difficulty of algorithms. It lets us know that, there are some algorithms that don’t exist, that sometimes, all we can do is guess at a problem, and never know if we’ve solved it. If we didn’t have the halting problem, we would still be searching for Hilbert’s magical algorithm which inputs theorems and outputs whether they’re true or not. Now we know we can stop looking, and we can put our efforts into finding heuristics and second-best methods for solving these problems.
Our family pictures were scattered over several computers, each with a unique photo management application. In an effort to get a good backup in place. I moved all my pictures to one computer where I accidentally deleted them. (Long story.) I was able to recover them all, but I had numerous duplicates and huge amounts of other image junk. To make matters much more complicated. I accidentally moved all files into one directory with super-long file names that represented their paths. (Another long story.) Yes, I should have built backups. Lesson learned. In any case, while scripting super-powers can sometimes get you into trouble, the only way to get out of them is with some good scripts.
We have decided to use Lightroom on Windows as a photo-management application. Our windows machines have a huge amount of storage that we can build out quickly with cheap hard drives. However, you can imagine one problem I have to solve is to eliminate a huge amount of duplicate images at different size, and to get rid of junk images.
Removing duplicates
I wrote the following code in Matlab to find duplicates and bin very dark images. It scans the directory for all images, reduces their size, computes an image histogram, which it then wraps into 16 sections, that are summed and normalized. I then run a 2-d correlation coefficient on each possible combination.
$$
r = \frac{
\sum_m \sum_n \left(A_{mn} – \hat A \right) \left(B_{mn} – \hat B \right)
}{
\sqrt{
\left(
\sum_m \sum_n \left(A_{mn} – \hat A \right)^2
\right)
\left(
\sum_m \sum_n \left(B_{mn} – \hat B \right)^2
\right)
}
}
$$
The result are comparisons such as this one.
And a histogram of the correlation coefficients shows a high degree of correlation in general.
My goal is to use this to keep the biggest photo and put the others in a directory of the same name. More to come, after I get some help.
As computers become powerful and solve more problems, the possibility that computers could evolve into a capability that could rise up against us and pose an existential threat is of increasing concern. After reading a recent book on artificial intelligence (AI), Superintelligence, Elon Musk recently said:
I think we should be very careful about artificial intelligence. If I were to guess like what our biggest existential threat is, it’s probably that. So we need to be very careful with the artificial intelligence. Increasingly scientists think there should be some regulatory oversight maybe at the national and international level, just to make sure that we don’t do something very foolish. With artificial intelligence we are summoning the demon. In all those stories where there’s the guy with the pentagram and the holy water, it’s like yeah he’s sure he can control the demon. Didn’t work out
This is a tough claim to evaluate because we have little understanding of how the brain works and even less understanding of how current artificial intelligence could ever lead to a machine that develops any sense of self-awareness or an original thought for that matter. Our very human minds use our imagination to fill in the gaps in our understanding and insert certainty where it doesn’t belong. While “dangerous” AI is a future hypothetical that no-one understands, there is no shortage of experts talking about it. Nick Bostrom, the author of Superintelligence, is a Professor, Faculty of Philosophy & Oxford Martin School; Director, Future of Humanity Institute; Director, Programme on the Impacts of Future Technology; University of Oxford. Musk, one of the most admired futurists and businessmen today, is joined by other thought-leaders such as Ray Kurzweil and Stephen Hawking in making statements such as: “Artificial intelligence could be a real danger in the not-too-distance future. It could design improvements to itself and outsmart us all.”
Bostrom gives us a name for this hypothetical goblin: Superintelligence. He defines it this as “an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.” While we can all expect that the capability and interconnectedness of computers will continue to increase, it is Bostrom’s use of the word intellect that causes the most controversy.
Can an intellect arise from basic materials and electricity? While this question has theological implications, this seems a possibility for many today and is in some sense a consequence of using evolution to form a complete worldview. When our current fascination with monism and Darwinism is combined with a growing awareness of that our reliance on and the capability of machines is growing geometrically, we are primed to accept Bostrom’s reductionist and materialist statement:
Biological neurons operate at a peak speed of about 200 Hz, a full seven orders of magnitude slower than a modern microprocessor (~2 GHz).*
How could we mere humans ever compete? If we accept the brain and consciousness are merely the result of chemical, electrical and mechanical processes than it ought to be emulable by synthetic materials. While this would require breakthroughs in 3D printing, AI and chemistry, the question for a modern materialist is not if this is possible, but when it will occur. The argument might go: while we don’t have AI like this now, super-intelligent machines could evolve much faster than us and may consequently find little use for the lordship of an inferior species.
If some experts think this way, when do they think human intelligence and capability are likely to be surpassed? The short answer is that they don’t agree on a timeline, but there is a slight consensus that computers will be able to match human intelligence. In 2006 a survey was conducted at the AI@50 conference and showed that 18{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} of attendees believed machines could “simulate learning and every other aspect of human intelligence” by 2056. Otherwise, attendees were split down the middle: 41{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} of attendees expected this to happen later and 41{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} expected machines to never reach that milestone. Another survey, by Bostrom, looked at the 100 most cited authors in AI in order to find the median year by experts expected machines “can carry out most human professions at least as well as a typical human”. From his survey, 10{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} said 2024, 50{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} said 2050, and 90{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} said to expect human-like intelligence in 2070. His summary is that leading AI researchers place a 90{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} probability on the development of human-level machine intelligence by between 2075 and 2090. While his question shapes the results, and the results say nothing about a general or a self-aware machine, he clearly has some experts agreeing with him.
But many others don’t agree. The strongest argument against self-awareness is that the intelligence of machines cannot be compared to human intelligence, because of a difference in purpose and environment. We process information (and exist) for different reasons. Currently, AI is custom built to accomplish (optimize) a series of tasks — and there is no reason to assume an algorithm would automatically transition to excel at another task. Machines are not dissatisfied, and harbor no resentment. Freedom is not an objective ideal for machines. Computers can only recognize patterns and run optimization algorithms. No current technology has shown any potential to develop into self-aware thought.
In the ninetieth century, Ada Lovelace speculated that future machines, no matter how powerful, would ever truly be a “thinking” machine. Alan Turing called this “Lady Lovelace’s objection” and responded with his basic turing test (can a human distinguish between human and computer-generated answers?) and predicted that computers would achieve this within a few decades. Sixty years later, we are not even close and we still haven’t seen anything like an original thought from a computer. John Von Neumann was fascinated by artificial intelligence, and realized that the architecture of the human brain was fundamentally different than any machine. Unlike a digital computer, the brain is an analog system that processes data simultaneously in mysterious ways. Von Neumann writes:
A new, essentially logical, theory is called for in order to understand high-complication automata and, in particular, the central nervous system. It may be, however, that in this process logic will have to undergo a pseudomorphosis to neurology to a much greater extent than the reverse.
That still hasn’t happened. Current technology isn’t even moving in the direction of original thought. Chess winning Deep Blue and Jeopardy! winning Watson won by quickly processing huge sets of data. Kasparov wrote after his loss to Deep Blue: “Deep Blue was only intelligent the way your programmable alarm clock is intelligent.”* The IBM research team that built Watson agrees that Watson had no degree of understanding of the questions it answered:
Computers today are brilliant idiots. They have tremendous capacities for storing information and performing numerical calculations—far superior to those of any human. Yet when it comes to another class of skills, the capacities for understanding, learning, adapting, and interacting, computers are woefully inferior to humans; there are many situations where computers can’t do a lot to help us. *
In fact, the current direction of technology might be going the opposite direction from self-awareness. According to Tomaso Poggio, the Eugene McDermott Professor of Brain Sciences and Human Behavior at MIT:
These recent achievements have, ironically, underscored the limitations of computer science and artificial intelligence. We do not yet understand how the brain gives rise to intelligence, nor do we know how to build machines that are as broadly intelligent as we are.*
Because we don’t understand how self-aware thought develops, all we have is a fleeting mirage in the future telling us that super-intelligence might be right around the corner. Without real-science, the only data to show us the future comes from our imagination and science fiction.
However, this might change. Betting against the ability for technology to accomplish any task is a bad idea. Tim Berners-Lee makes a reasonable argument when he says, “We are continually looking at the list of things machines cannot do – play chess, drive a car, translate language – and then checking them off the list when machines become capable of these things. Someday we will get to the end of the list.”*
Currently IBM and Qualcomm are building chips patterned after neurological processes and they are developing new software tools that simulate brain activity. By modeling the way individual neurons convey information, developers are currently writing and compiling biologically inspired software. The Neuromorphic Computing Platform from the European Union currently incorporates 50*106 plastic synapses and 200,000 biologically realistic neuron models on a single 8-inch silicon wafer. Like a natural system, they do not pre-program any code but only use logic that “evolves according to the physical properties of the electronic devices”.*
Should such a project produce self awareness, how dangerous would this be when compared to other existential threats? The future will clearly have higher interconnectivity and greater dependence on machines and they will continue to become more capable. In The Second Machine Age, I agree with Erik Brynjolfsson and Andrew McAfee when they write:
Digital technologies—with hardware, software, and networks at their core—will in the near future diagnose diseases more accurately than doctors can, apply enormous data sets to transform retailing, and accomplish many tasks once considered uniquely human.
Any time there is a great deal of interdependency, there is also a great deal of systemic risk. This will apply to our transportation networks, healthcare, and military systems and is a particular problem if we can’t build much more secure software. However, the threat here is malicious use combined with vulnerable software, not rouge AI. In this context, AI is most dangerous in its ability to empower a malicious actor. If, in the future, our computers are defended automatically by computers then a very powerful AI will be best equipped to find vulnerabilities, build exploits and conduct attacks. AI will also be critical to innovation and discovery as both humans and computers collaborate on societies’ hardest problems. To be most ready for this capability, the best strategy is to have the best AI which is only possible from a well-funded, diverse and active research base.
However, what if science develops a superior artificial intellect? Waiting to pull the power-cord is not a wise strategy. Issac Assimov provided us with three laws to follow to ensure benevolent interactions between humanity and machines:
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Clearly, military systems will be developed which don’t follow these laws. While they have pervaded science fiction and are referred to in many books, films, and other media, they do little to guide a national strategy towards protecting us from rouge AI. Bostrom proposes regulatory approaches such as pre-programming a solution to the “control problem” of how to prevent the superintelligence from wiping out humanity or instilling the superintelligence with goals that are compatible with human survival and well-being. He also proposes research be guided and managed within a strict ethical framework. Stephen M. Omohundro writes that intelligent systems will need to be carefully designed to prevent them from behaving in harmful ways and proposes developing a universal set of values in order to establish a “friendly AI”.
The Machine Intelligence Research Institute has the mission of ensuring that the creation of smarter-than-human intelligence has a positive impact. They are conducting research to ensure computers can reason coherently about their own behavior and are consistent under reflection, trying to formally specify an AI’s goals in order to ensure such that the formalism matches their designer’s intentions and considering how to ensure those intended goals are preserved even as an AI modifies itself. These are worthy and interesting research goals, but if this science is formally developed, it will only ensure that benevolent designers will produce safe systems.
These approaches fail to consider the difficulty of accounting for unintended consequences that occur when goals are translated into machine-implementable code. A strategy that relies on compliance also fails to account for malicious actors. We should develop the best and most diverse AI possible to both protect us from malicious activity wether it is human directed or not. Such a strategy accounts for Bostrom’s main point that the first superintelligence to be created will have decisive first-mover advantage and, in a world where there is no other system remotely comparable, it will be very powerful. Only a diverse array of AI could counter such a threat.
Fortunately, since we are still dealing with a hypothetical, there is time to explore mitigation options as AI develops. I also agree with Bostrom’s Oxford University colleagues who suggest that nuclear war and the weaponization of biotechnology and nanotechnology present greater threats to humanity than superintelligence. For our lives, there is much greater danger of losing your job to a robot than losing your life. Perhaps the greatest threat is an over-reaction to AI development which prevents us from developing the AI needed to solve our hardest problems.
