wand wolken malen

wand wolken malen

david malan: all right, welcome back. before we dive into cloud computing,i thought i'd pause for a moment if there are any outstanding questionsor topics that came up during lunch that might now be of interest. >> audience: [inaudible] >> david malan: ok. oh, ok. audience: [inaudible] >> david malan: no, of course.

ok, well hopefully all of yourproblems arise in the next few hours and tomorrow especially. but let's take a look, then, at wherethe last discussion about setting up a website leads, more generallywhen it comes to cloud computing, setting up a server architecture,the kinds of decisions that engineers anddevelopers and managers need to make when it comesto doing more than just signing up for a $10 per month web hostwhen you actually want to build out your own infrastructure.

and we'll try to tie this back,for instance, to dropbox and others like them. >> so let's start to considerwhat problems arise as business gets good and good problems arise. so in the very simplest case of havingsome company that has a web server, you might have, let's say, a server thatwe'll just draw that looks like this. and these days, most servers-- and let'sactually put a picture to this just so that it's a little less nebulous. >> so dell rack server--back in the day, there

were mainframe computersthat took up entire rooms. these days, if you wereto get a server, it might look a little something like this. servers are measured in whatare called rack units, or rus. and one ru is 1.5 inches,which is an industry standard. so this looks like a two ru server. so it's 3 inches tall. and they're generally 19 inches wide,which means all of this kind of stuff is standardized.

>> so if you look in a data center--not just at one server, but let's take a look at google'sdata center and see if we see a nice picture in google images. this is much better lit than youwould typically find, and much sexier looking as a result. butthis is what looks like a couple hundred servers allabout that same size, actually, in rack after rack afterrack after rack in a data center. >> something like this-- this may wellbe google's, since i googled google's. but it could be representativeof more generally

a data center in which manycompanies are typically co-located. and co-located generally meansthat you go to a place like equinix or other vendors that have largewarehouses that have lots of power, lots of cooling, hopefullylots of security, and individual cages enclosing racks ofservers, and you either rent the racks or you bring the racks in. >> and individual companies,startups especially, will have some kind of biometricsto get into their cage, or a key, or a key card.

you open up the door. and inside of there is justa square footage footprint that you're paying for, inside ofwhich you can put anything you want. >> and you typically pay for the power. and you pay for the footprints. and then you payyourself for the servers that you're bringing into that space. and what you then have theoption to do is pay someone for your internet service connectivity.

you can pay any numberof vendors, all of whom typically come into that data center. >> but the real interesting question is,what actually goes in those racks? they might all very welllook like what we just saw. but they perform different functionsand might need to do different things. and let's actuallymotivate this discussion with the question of, what problemstarts to arise if you're successful? >> so you've got a websitethat you've built. and maybe it sells widgetsor something like that.

and you've been doing very wellwith sales of widgets online. and you start to experiencesome symptoms, your website. what might be some ofthe technical symptoms that users report as businessis growing and booming and your website isbenefiting from that? >> david malan: yeah, exactly. so you might have aslowdown of your website. and why might that happen? well, if we assume, forthe sake of discussion

right now, that you're on oneof these commercial web hosts that we talked about before lunch,that you pay some number of dollars to per month, and you've already paidfor the annual cost of your domain name, that web host is probablyoverselling their resources to some extent. so you might have a usernameand password on their server. but so might several other, or severaldozen other, or maybe even several hundred other, users. >> and websites live physicallyon the same server.

why is this possible? well these days, serverslike this typically have multiple hard drives, maybeas many as six or more hard drives, each of which might be as muchas 4 terabytes these days. so you might have 24 terabytes of spacein just one little server like this. >> and even if you steal some of that spacefor redundancy, for backup purposes, it's still quite a lot of space. and certainly, a typical websitedoesn't need that much space. just registering usersand storing logs of orders

doesn't take all that much space. so you can partition it quitea bit and give every user just a little slice of that. >> meanwhile, a computerlike this these days typically has multiple cpus-- not justone, maybe two, maybe four, maybe 16, or even more. and each of those cpushas something called a core, which is kind of likea brain inside of a brain. so in fact most everyone here withmodern laptops has probably a dual core

or quad core cpu-- and probably onlyone cpu inside of a laptop these days. but desktop computersand rack computers like this might have quite a fewmore cpus, and in turn cores. >> and frankly, even in our macs and pcs oftoday, you don't really need dual cores or quad cores to check your email. if there's any bottleneck whenit comes to using a computer, you the human are probably theslowest thing about that computer. and you're not going to be able tocheck your email any faster if you have four times as many cpus or cores.

>> but the same is kindof true of a server. one single website might notnecessarily need more than one cpu or one core, onesmall brain inside doing all of the thinking and the processing. so manufacturers have similarlystarted to slice up those resources so that maybe your website gets onecore, your website gets one core, or maybe we're sharing one such core. we're also sharing disk space. and we're also sharing ram,or random access memory

from before, of whichthere's also a finite amount. >> and that's the key. no matter how expensivethe computer was, there's still a finiteamount of resources in it. and so the more and more youtry to consume those resources, the slower things might become. but why? why would things slow down as asymptom of a server being overloaded? what's happening?

david malan: yeah, exactly. i proposed earlier thatram is a type of memory. it's volatile, whereby that'swhere apps and data are stored when they're being used. and so therefore there'sonly a finite number of things you can apparently do at once. and it's also faster,which is a good thing. but it's also more expensive,which is a bad thing. and it's also therefore present in lowerquantities than disk space, hard disk

space, which tends to be cheaper. >> in other words, youmight have 4 terabytes of disk space in your computer. but you might have 4gigabytes, or 64 gigabytes, in order of magnitude, a factor of1,000 less, of ram in your computer. so what does a computer do? well, suppose that youdo have 64 gigabytes of ram in a server like this, whichwould be quite common, if not low these days.

but suppose you have so manyusers doing so many things that you kind of sort ofneed 65 gigabytes of memory to handle all of thatsimultaneous usage? >> well, you could just say,sorry, some number of users just can't access the site. and that is the measureof last resort, certainly. or you, as the operatingsystem, like the windows or mac os or linux or solaris or anynumber of other oses on that server, could just decide, you know what?

i only have 64 gigabytes of ram. i kind of need 65. so you know what? i'm going to take 1 gigabyteworth of the data in ram that was the least recently accessedand just move it to disk temporarily, literally copy it from the fastmemory to the slower memory so that i can then handle that65th gigabyte need for memory, do some computation on it. then when i'm done doing that,i'll just move that to disk,

move that other ram i temporarily puton disk back into the actual hardware so that i'm kind of multitasking. >> so i'm sort of putting thingstemporarily in this slower space so i create the illusionof handling everyone. but there's a slowdown. why? well, inside of these harddisks these days is what? rather, what makes a harddrive different from ram as best you know now?

>> david malan: ok, true. >> david malan: so very true. and that is a side effect or featureof the fact that ram is indeed faster. and therefore you want touse it for current use. and a disk is slower. but it's permanent, or nonvolatile. so you use it for long term storage. but in terms ofimplementation, if i look up what's called a dimm, dual inline memorymodule, this is what a piece of ram

might typically look like. >> so inside of our mac-- that's a bug. inside of our macs and pcs, our desktopcomputers would have sticks of memory, as you would call them,or dimms, or simms back in the day, of memorythat look like this. our laptops probably have things thatare a third the size or half the size. they're a little smaller,but the same idea-- little pieces of green siliconwafer or plastic that has little black chips on them with lotsof wires interconnecting everything.

you might have a whole bunch ofthese inside of your computer. but the takeaway here isit's entirely electronic. there's just electronsflowing on this device. by contrast, if we look atthe inside of a hard drive and pull up a picturehere, you would instead see something like this,which does have electricity going through it ultimately. but what also jumps outat you about this thing? david malan: yeah, there'sapparently moving parts.

it's kind of like an old recordplayer or phonograph player. and it pretty much is. it's a little fancier than that--whereas a phonograph player used grooves in the record, this actuallyuses tiny little magnetic particles that we can't quite see. but if a little magnetic particlelooks like this, it's considered a 1. and if it looks like this,north-south instead of south-north, it might be a 0. and we'll see tomorrow how we can buildfrom that to more interesting things.

>> but anything that'sgot to physically move is surely going to go slowerthan the speed of light, which in theory is whatan electron might flow at, though realistically not quite. so mechanical devices-- much slower. but they're cheaper. and you can fit so muchmore data inside of them. so the fact that thereexists in the world something called virtual memory,using a hard disk like this

as though it were ramtransparent to the user, simply by moving datafrom ram to the hard disk, then moving it back when you needit again, creates the slowdown. because you literally have tocopy it from one place to another. and the thing you're copying it to andfrom is actually slower than the ram where you want it to be. >> the alternative solution here--if you don't like that slow down, and your virtual memory issort of being overtaxed, what's another solution to this problem?

david malan: well,increasing the virtual memory would let us do this onan even bigger scale. we could handle 66 gigabytes worthof memory needs, or 67 gigabytes. but suppose i don't likethis slow down, in fact i want to turn off virtualmemory if that's even possible, what else could i throw atthis problem to solve it, where i want to handle more usersand more memory requirements than i physically have at the moment? >> david malan: unfortunately no.

so the cpu and the cores they'rein are a finite resource. and there's no analog in that context. good question, though. so just to be clear, too, ifinside of this computer is, let's say, a stick of ram that lookslike this-- and so we'll call this ram. and over here is the hard disk drive. and i'll just draw thispictorially as a little circle. there are 0's and 1's in both ofthese-- data, we'll generalize it as. >> and essentially, if a user isrunning an application like,

let's say, a website that requires thismuch ram per user, what i'm proposing, by way of this thingcalled virtual memory, is to just temporarily movethat over here so that now i can move someone else's memoryrequirements over there. and then when that's done,i can copy this back over and this goes here, thereby movingwhat i wanted in there somewhere else altogether. >> so there's just a lot ofswitcheroo, is the takeaway here. so if you don't like this, and you don'twant to put anything on the hard drive,

what's sort of the obviousbusiness person's solution to the problem, or the engineer'ssolution, for that matter, too? >> david malan: yeah, i mean literallythrow money at the problem. and actually, this is the perfectsegue to some of the higher level discussions of cloud computing. because a lot of it is motivatedby financial decisions, not even necessarily technological. if 64 gigs of ram is too little, well,why not get 128 gigabytes of ram? why not get 256 gigabytes of ram?

well, why not? david malan: well, itcosts more money, sure. and if you already have sparehard disk space, effectively, or equivalently, hard disk space is somuch cheaper you might as well use it. so again, there's this trade off thatwe saw even earlier on this morning, where there's really notnecessarily a right answer, there's just a better or worse answerbased on what you actually care about. >> so there's also technological realities. i cannot buy a computer,to my knowledge,

with a trillion gigabytesof ram right now. it just physically doesn't exist. so there is some upper bound. but if you've ever even shoppedfor a consumer mac or pc, too, generally there'sthis curve of features where there might be a good,a better, and a best computer. >> and the marginal returnson your dollar buying the best computer versusthe better computer might not be nearly as highas spending a bit more money

and getting the better computerover the good computer. in other words, you're paying apremium to get the top of the line. >> and what we'll see in thediscussion of cloud computing is that what's very common thesedays, and what companies like google early on popularized, was not payingfor and building really fancy, expensive souped up computers withlots and lots of everything, but rather buying or building prettymodest computers but lots of them, and using something that's generallycalled horizontal scaling instead of vertical scaling.

>> so vertical scaling would mean get moreram, more disk, more of everything, and sort of investvertically in your hardware so you're just getting thebest of the best of the best, but you're paying for it. horizontal scaling is sort of get thebottom tier things, the good model, or even the worse model,but get lots of them. but as soon as you get lots ofthem-- for instance, in this case, web servers, if this one serveror one web host is insufficient, then just intuitively, thesolution to this problem of load

or overload on your serversis either get a bigger server or, what i'm proposing here insteadof scaling vertically so to speak, would be, you know what? just get a second one of these. or maybe even get a third. but now we've createdan engineering problem by nature of this businessor financial decision. what's the engineering problem now? >> david malan: yeah, how doyou connect them and-- sorry?

david malan: right,because i still have-- if i reintroduce me into this picture,if this is my laptop somewhere on the internet, which is now betweenme and the company we're talking about, now i have to figure out, to whichserver do i send this particular user? and if there's other users, likethis, and then this one over here, and maybe this is user a, thisis user b, this is user c, and this is server 1, 2, and 3-- nowan intuitive answer might here be just, we'll send user a to 1and b to 2 and c to 3. and we can handle 3 times as many users.

>> but that's an oversimplification. how do you decide whom to send where? so let's try to reason through this. so suppose that computersa, b, and c are customers, and servers 1, 2, and 3 arehorizontally scaled servers. so they're sort of identical. they're all running the same software. and they can all do the same thing. but the reason we havethree of them is so

that we can handle threetimes as many people at once. >> so we know from ourdiscussion prior to lunch that there's hardware in betweenthe laptops and the servers. but we'll just sort of generalizethat now as the internet or the cloud. but we know that in my home,there's probably a home router. near the servers, there's probablya router, dns server, dhcp. there can be anythingwe want in this story. >> so how do we start to decide,when user a goes to something.com, which server to route the user to?

how might we begin to tell this story? audience: load balancing? david malan: load balancing. what do you mean by that? >> audience: returningwhere the most usage is and which one has themost available resources. david malan: ok, so let meintroduce a new type of hardware that we haven't yet discussed, whichis exactly that, a load balancer. this too could just be a server.

it could look exactly likethe one we saw a moment ago. a load balancer really isjust a piece of software that you run on a piece of hardware. >> or you can pay a vendor, likecitrix or others, cisco or others. you can pay for their own hardware,which is a hardware load balancer. but that just means theypre-installed the load balancing software on their hardware andsold it to you all together. so we'll just draw it as arectangle for our purposes. >> how now do i implement a load balancer?

in other words, when user a wants tovisit my site, their request somehow or other, probably by way of thoserouters we talked about earlier, is going to eventually reachthis load balancer, who then needs to make a routing-like decision. but it's routing for sortof a higher purpose now. it's not just about gettingfrom point a to point b. it's about deciding whichpoint b is the best among them-- 1, 2, or 3 in this case. >> so how do i decide whetherto go to 1, to 2, to 3?

what might this black box, so tospeak, be doing on the inside? this too is another example incomputer science of abstraction. i have literally drawn a load balanceras a black box in black ink, inside of which is some interestinglogic, or magic even, out of which needs to comea decision-- 1, 2, or 3. and the input is just a. david malan: i'm sorry? david malan: all right, how might wecategorize the types of transactions here?

>> audience: viewing a webpageversus querying a database. david malan: ok, that's good. so maybe this user awants to view a web page. and maybe it's even static content,something that changes rarely, if ever. and that seems like apretty simple operation. so maybe we'll just arbitrarily,but reasonably, say, server 1, his purpose in life isto just serve up static content, files that rarely, if ever, change. maybe it's the images on the page.

maybe it's the text on the page orother such sort of uninteresting things, nothing transactional, nothing dynamic. >> by contrast, if user a is checkingout of his or her shopping cart that requires a database, someplace to storeand remember that transaction, well maybe that requestshould go to server 2. so that's good. so we can load balance basedon the type of requests. how else might we do this? what other--

>> audience: based on the server'sutilization and capacity. david malan: right, ok. so you mentioned that earlier, kareem. so what if we provide some inputon [inaudible] among servers 1, 2, and 3 to this load balancer so thatthey're just constantly informing the load balancer what their status is? like, hey, load balancer,i'm at 50% utilization. in other words, i havehalf as many users as i can actually handle right now.

hey, load balancer, i'mat 100% utilization. hey, load balancer, 0% utilization. the load balancer, if it'sdesigned in a way that can take in those commentsas input, it can then decide, ooh, number 2 is at 100%. let me send no future requests to himother than the users already connected. this guy's at 0%. let's send a lot of traffic to him. this guy said he's at 50%.

let's send some traffic to him. >> so that would be an ingredient, thatwe could take load into account. and it's going to change over time. so the decisions will change. so that's a really good technique,one that's commonly used. what else could we do? and let's actually just summarize here. so the decisions here could beby type of traffic, i'll call it. it can be based on load.

let's see if we can'tcome up with a few other. david malan: location. so that's a good one. so location-- how might youleverage that information? >> david malan: oh, that's good. and about how many millisecondswould it decrease by based on what we saw thismorning, would you say? >> david malan: well, basedon the trace routes we saw earlier, which is justa rough measure of something,

at least how long it takesfor data to get from a to b feels like anything local was, what,like 74 milliseconds, give or take? and then anything 100 plus,200 plus was probably abroad. and so based on that alone,it seems reasonable to assume that for a user in the usto access a european server might take twice or three timesas long, even in milliseconds, than it might take if thatserver were located here geographically, or vice versa. so when i proposedearlier that especially

once you cross that 200 millisecondthreshold, give or take, humans do start to notice. and the trace route is justassuming raw, uninteresting data. when you have a website, you have toget the user downloading images or movie files, lots of text,subsequent requests. we saw when we visited, what wasit, facebook or amazon earlier, there's a whole lot of stuffthat needs to be downloaded. so that's going to add up. so multi-seconds mightnot be unreasonable.

so good, geography is one ingredient. so in fact companies likeakamai, if you've heard of them, or others have long takengeography into account. and it turns out that by nature of anip address, my laptop's ip address, you can infer, with some probability,where you are in the world. and in fact there'sthird party services you can pay who maintain databasesof ip addresses and geographies that with high confidence will betrue when asked, where in the world is this ip address?

>> and so in fact whatother companies use this? if you have hulu or netflix, ifyou've ever been traveling abroad, and you try to watch something onhulu, and you're not in the us, you might see a messagesaying, not in the us. sorry, you can't view this content. >> david malan: oh, really? but yes, so actually that'sa perfect application of something very technicalto an actual problem. if you were to vpn fromeurope or asia or anywhere

in the world to your corporateheadquarters in new york or wherever you are, you'regoing to create the appearance to outside websites thatyou're actually in new york, even though you'rephysically quite far away. >> now you the user are going toknow you're obviously away. but you're also going to feel it becauseof those additional milliseconds. that additional distance and theencryption that's happening in the vpn is going to slow things down. so it may or may notbe a great experience.

but hulu and netflix are going to seeyou as sitting somewhere in new york, as you've clearly gleaned. what a perfect solution to that. >> all right, so geography is one decision. what else might we use to decide howto route traffic from a, b, and c to 1, 2, and 3, again, puttingthe engineering hat on? this all sounds very complicated. uh, i don't even know whereto begin implementing those. give me something that's simpler.

what's the simplest wayto make this decision? >> audience: is the server available? >> david malan: is the server available? so not bad. that's good. that's sort of a nuancing of load. so let's keep that in the load category. if you're available, i'm justgoing to send the data there. but that could backfire quickly.

because if i use that logic, and if ialways ask 1, are you on, are you on, are you on, if the answer is always yes,i'm going to send 100% of the traffic to him, 0% to everyone else. and at some point, we're going to hitthat slowdown or site unavailable. so what's slightly better thanthat but still pretty simple and not nearly as clever as taking allthese additional data into account? >> audience: cost per server. david malan: cost per server. ok, so let me toss thatin the load category, too.

because what you'll find ina company, too-- that if you upgrade your serversover time or buy more, you might not be able to get exactlythe same versions of hardware. because it falls out of date. you can't buy it anymore. prices change. >> so you might have disparate serversin your cluster, so to speak. that's totally fine. but next year's hardwaremight be twice as fast,

twice as capable as this year's. so we can toss thatinto the load category. this feedback loop between 1,2, and 3 in the load balancer could certainly tell it,hey, i'm at 50% capacity. but by the way, i alsohave twice as many cores. use that information. even simpler-- and this is goingto be a theme in computer science. when in doubt, or when you want a simplesolution that generally works well over time, don't choose the sameserver all the time, but choose--

>> audience: a random one? david malan: --a random server. yeah, choose one or the other. so randomness is actuallythis very powerful ingredient in computer science,and in engineering more generally, especially when you wantto make a simple decision quickly without complicating it with allof these very clever, but also very clever, solutions that requireall the more engineering, all the more thought, whenreally, why don't i

just kind of flip a coin, or athree sided coin in this case, and decide whether to go 1, 2, 3? >> that might backfire probabilistically,but much like the odds of flipping heads again andagain and again and again and again and again is possible inreality-- super, super unlikely. so over time, odds arejust sending users randomly to 1, 2, and 3 is going towork out perfectly fine. and this is a techniquegenerally known as round robin. >> or actually, that's not round robin.

this would be the random approach. and if you want to be evena little simpler than that, round robin would be, first person goesto 1, second person to 2, third person to 3, fourth person to 1. and therein lies the round robin. you just kind of go around in a cycle. >> now, you should be smart about it. you should not blindly send the user toserver number one if what is the case? if it's at max capacity, orit's just no longer responsive.

so ideally you want somekind of feedback loop. otherwise, you just send allof your users to a dead end. but that can be taken into account, too. >> so don't under appreciate the value ofjust randomness, which is quite often a solution to these kinds of problems. and we'll write down round robin. so how do some companies implementround robin or randomness or any of these decisions? well unfortunately, theydo things like this.

let me pull up another quick screenshot. >> actually, let's do two. i don't know why we'regetting all of these dishes. that's very strange. all right, what i reallywant is a screenshot. that is weird. all right, so i can spoof this. i don't know how much fartheri want to keep scrolling. >> so very commonly, you'll find yourselfat an address like www.2.acme.com,

maybe www.3 or 4 or 5. and keep an eye for this. you don't see it that often. but when you do, it kind of tends tobe bigger, older, stodgier companies that technologically don't reallyseem to know what they're doing. and you see this on tech companiessometimes, the older ones. >> so what are they doing? how are they implementingload balancing, would it seem? if you find yourself as theuser typing www.something.com,

and suddenly you're atwww.2.something.com, what has their loadbalancer probably done? >> david malan: yeah, so theload balancer is presumably making a decision based on one ofthese decision making processes-- doesn't really matter which. but much like i've drawn thenumbers on the board here, the servers aren't justcalled 1, 2, and 3. they're probably calledwww1, www2, www3. and it turns out that inside ofan http request is this feature.

and i'm going tosimulate this as follows. >> i'm going to open up that samedeveloper network tab as before just so we can see what's goingon underneath the hood. i'm going to clear the screen. and i'm going to go to, let'ssay, http://harvard.edu. now for whateverbusiness reasons, harvard has decided, like many,many other websites, to standardize itswebsite on www.harvard.edu for both technicaland marketing reasons.

it's just kind of invogue to have the www. >> so the server at harvard hasto somehow redirect the user, as i keep saying, fromone url to the other. how does that work? well, let me go ahead and hit enter. and notice the url indeed quicklychanged to www.harvard.edu. let me scroll back in thishistory and click on this debug diagnostic information, if you will. let me look at my request.

>> so here's the request i made. and notice it's consistent with the kindof request i made of facebook before. but notice the response. what's different inthe response this time? >> david malan: yeah, so it's not a 200 ok. it's not a 404 not found. it's a 301 moved permanently, whichis kind of a funny way of saying, harvard has upped and movedelsewhere to www.harvard.edu. the 301 signifies thatthis is a redirect.

and to where should the userapparently be redirected? there's an additional tidbit ofinformation inside that envelope. and each of these lines will nowstart calling an http header. header is just a key valuepair-- something colon something. it's a piece of information. where should the newlocation apparently be? notice the last lineamong all those headers. >> david malan: yeah, so there'sadditional information. the first line that i've highlightedsays 301 moved permanently.

well, where has it moved? the last line-- and they don'thave to be in this order. it can be random. location colon means, heybrowser, go to this url instead. >> so browsers understand http redirects. and this is a very, verycommon way of bouncing the user from one place to another. for instance, if you've ever triedto visit a website that you're not logged into, you might suddenly findyourself at a new url altogether being

prompted to log in. >> how does that work? the server is probably sending a 301. there's also other numbers, like302, somewhat different in meaning, that send you to another url. and then the server,once you've logged in, will send you back to whereyou actually intended. >> so what, then, are poorlyengineered websites doing? when you visitwww.acme.com, and they just

happen to have named their serverswww1, www2, www3, and so forth, they are very simply--which is fair, but very sort of foolishly-- redirecting you toan actually differently named server. and it works perfectly fine. it's nice and easy. >> we've seen how it would bedone underneath the hood in the virtual envelope. but why is this arguably abad engineering decision? and why am i sort of condescendingtoward this particular engineering

approach? argue why this is bad. ben? david malan: each server would have tohave a duplicate copy of the website. i'm ok with that. and in fact, that's what i'msupposing for this whole story, since if we wanted-- wellactually, except for dan's earlier suggestion, where if you have differentservers doing different things, then maybe they could actually befunctionally doing different things.

>> but even then, at some point, yourdatabase is going to get overloaded. your static assets serveris going to get overloaded. so at some point, we'reback at this story, where we need multiple copies of the same thing. so i'm ok with that. >> david malan: ok, so some pagesmight be disproportionately popular. and so fixating on one addressisn't necessarily the best thing. [inaudible]? >> david malan: what do you mean by that?

so you don't want tonecessarily have-- you certainly don't want to have your usersmanually typing in www1 or www2. from a branding perspective, itjust looks a little ridiculous. if you just want sort of aclean, elegant experience, having these sort of randomlynumbered urls really isn't good. because then users are surelygoing to copy and paste them into emails or instant messages. >> now they're propagating. now you're sort of confusing yourless technical audience, who thinks

your web address is www2.something.com. there's no compelling semantics to that. it just happens to be an underlyingtechnical detail that you've numbered your servers in this way. >> and worse yet, what if, for instance,maybe around christmas time when business is really booming,you've got www1 through www99, but in january and february andonward, you turn off half of those so you only have www1 through www50? what's the implication now for thatvery reasonable business decision?

david malan: you need tomanage all of those still. david malan: exactly. that's kind of the catch there. if your customers are in the habit ofbookmarking things, emailing them, just saving the url somewhere, orif it's just in their auto complete in their browser so they'renot really intentionally typing it, it's just happening, they might,for 11 months out of the year effectively, reach a dead end. and only the most astute ofusers is going to realize,

maybe i should manuallyremove this number. i mean, it's just not going to happenwith many users, so bad for business, bad implementation engineering wise. >> so thankfully, it's not even necessary. it turns out that whatload balancers can do is instead of saying, when amakes a request-- hey a, go to 1. in other words, insteadof sending that redirect such that step one in thisprocess is the go here, he is then told to go elsewhere.

and so step three is, he goes elsewhere. >> you can instead continue to route, tokeep using that term, all of a's data through the load balancer so that henever contacts 1, 2, or 3 directly. all of the traffic does get "routed"by the load balancer itself. and so now we're sort ofdeliberately blurring the lines among these various devices. a load balancer can route data. it's just a function that it has. >> so a load balancer, too, it'sa piece of software, really.

and a router is a piece of software. and you can absolutely havetwo pieces of software inside of one physical computer so a loadbalancer can do these multiple things. >> so there's one other wayto do this, which actually goes back to sort of first principlesof dns, which we talked about before break. dns was domain name system. remember that you canask a dns server, what's the ip address ofgoogle.com, facebook.com?

>> and we can actually do this. a tool we did not use earlier isone that's just as accessible, called nslookup, for name server lookup. and i'm just going to type facebook.com. and i see that facebook's ipaddress is apparently this. let me go ahead and copythat, go to a browser, and go to http:// and thatip address and hit enter. and sure enough, it seems to work. >> now working backwards, what wasinside of the virtual envelope

that facebook responded with wheni visited that ip address directly? because notice, where am i now? where am i now, the address? >> david malan: at the secure version,and at the www.facebook.com. so it's not even justthe secure ip address. facebook has taken it upon itselfto say, this is ridiculous. we're not going to keep you at thisugly looking url that's numeric. we're going to send you an httpredirect by way of that same header that we saw before--location colon something.

>> and so this simply means that underneaththe hood is still this ip address. every computer on the internethas an ip address, it would seem. but you don't necessarily haveto expose that to the user. and much like back in the day, therewas 1-800-collect, 1-800-c-o-l-l-e-c-t, in the us, was a way of making collectcalls via a very easily memorable phone number, or 1-800-mattress to buy a bed,and similar mnemonics that you even see on the telephone kind of sort ofstill, that letters map to numbers. >> now, why is that? well, it's a lot easier to memorize1-800-mattress or 1-800-collect instead

of 1-800 something something somethingsomething something something something, where eachof those is a digit. similarly, the world learnedquickly that we should not have people memorize ip addresses. that would be silly. we're going to use names instead. and that's why dns was born. >> all right, so with that said, in termsof load balancing, let's try yahoo.com. well, that's interesting.

yahoo seems to be returning three ips. so infer from this,if you could, what is another way that we could implementthis notion of load balancing maybe without even using a physicaldevice, this new physical device? >> in other words, can i take away thefunding you have for the load balancer and tell you to use some existingpiece of hardware to implement this notion of load balancing? and the spoiler is,yes, but what, or how? what is yahoo perhaps doing here?

kareem? ok, chris? david malan: yeah, allthree of those work. so randomness, round robin,location-- you can just leverage an existing piece of the puzzlethat we talked about earlier of the dns system and simply say, when the firstuser of the day requests yahoo.com, give them the first ip address,like the one ending in 45 up there. and the next time a user requeststhe ip address of yahoo.com from somewhere in the world,give them the second ip,

then the third ip, then thefirst ip, then the second. or be smart about itand do it graphically. or do it randomly and not just doit round robin in this fashion. >> and in this case, thenwe don't even need to introduce this blackbox into our picture. we don't need a new device. we're simply telling computersto go to the servers directly, effectively, but notby way of their name. they never need to know the name.

they're just being told that yahoo.commaps to any one of these ip addresses. >> so it sends the exact same request. but on the outside ofthe envelope, it simply puts the ip that it was informed of. and in this way, too, couldwe load balance the requests by just sending the envelope to adifferent one of yahoo's own servers? >> and if we keep digging, we'll seeprobably other companies with more. cnn has two publicly exposed. though actually if we do this againand again-- cnn.com-- you can see

they're changing order, actually. so what mechanism iscnn using, apparently? >> audience: random. david malan: well, itcould be random, though it seems to be cycling back and forth. so it's probably round robin wherethey're just switching the order so that i'll presumably take the first. my computer will takethe first each time. so that's load balancing.

and that allows us, ultimately,to map data, or map requests, across multiple servers. so what kinds ofproblems now still exist? it feels like we just reallysolved a good problem. we got users to different servers. but-- oh, and chris, didyou have a question before? >> david malan: totally depends. so what is happening here? and we can actually see this.

so let's try yahoo's. actually, let's go to facebook. because we know that one works. so i'm going to copythat ip address again. i'm going to close all these tabs. i'm going to go open thatspecial network tab down here. and i'm going to visit only http://. and now i'm going to hit enter. and let's see what happened.

>> if i look at that request, noticethat my-- facebook is a bad example. because they have asuper fancy technique that hides that detail from us. let me use yahooinstead-- http:// that ip. let's open our networktab, preserve log. and here we go, enter. that's funny. ok, so here is the famed 404 message. what's funny here is that theyprobably never will be back.

because there's probablynot something wrong per se. they have just deliberatelydecided not to support the numeric form of their address. >> so what we're actually seeing in thenetwork tab, if i pull this up here, is, as i say, the famed 404, whereif i look at the response headers, this is what i got here-- 404 not found. so let's try one other. let's see if cnn cooperates with us. i'll grab one of cnn's ip addresses,clear this, http, dah, dah, dah, dah.

so in answer to chris'squestion, that one worked. >> and let's go to response headers. actually no, all right, i amstruggling to find a working example. so cnn has decided, we'll just leave youat whatever address you actually visit, branding issues aside. but what wouldn't be happening, ifwe could see it in facebook's case, is we would get a 301 movedpermanently, most likely, inside of which islocation:https://www.facebook.com. and odds are www.facebook.com is analias for the exact same server we just

went to. >> so it's a little counterproductive. we're literally visiting the server. the server is then telling us, go away. go to this other address. but we just so happen to begoing back to that same server. but presumably we now stay on thatserver without this back and forth. because now we're using the namedversion of the site, not the numeric. good question.

>> ok, so if we now assume-- wehave solved load balancing. we now have a mechanism,whether it's via dns, whether it's via this black box, whetherit's using any of these techniques. we can take a user's request in andfigure out to which server, 1, 2, or 3, to send him or her. >> what starts to break about our website? in other words, we havebuilt a business that was previously on one single server. now that business is runningacross multiple servers.

what kinds of assumptions,what kinds of design decisions, might now be breaking? >> this is less obvious. but let's see if we can't put ourfinger on some of the problem we've created for ourselves. again, it's kind of like holdingdown the leak in the hose. and now some new issuehas popped up over here. david malan: ok, so we have tokeep growing our hard disk space. i'm ok with that right now.

because i think i canhorizontally scale. like if i'm running low, i'll just geta fourth server, maybe a fifth server, and then increase our capacityby another 30% or 50% or whatnot. so i'm ok with that, at least for now. david malan: ok, so that's a good point. so suppose the serversare not identical. and customer serviceor the email equivalent is getting some message from a usersaying, this isn't working right. it's very possible, sometimes,that maybe one or more servers

is acting a bit awry, but notthe others, which can certainly make it harder to chase down the issue. you might have to look multiple places. >> that is manifestationof another kind of bug, which is that you probably shouldhave designed your infrastructure so that everything is truly identical. but it does reveal a new problemthat we didn't have before. what else? >> david malan: yeah,there's more complexity.

there's physically more wires. there's another device. in fact, i've introduced a fundamentalconcept and a fundamental problem here known as a single pointof failure, which, even if you've never heardthe phrase, you can probably now work backwards and figure it out. what does it mean that i have a singlepoint of failure in my architecture? and by architecture, i justmean the topology of it. >> david malan: yeah, what ifthe load balancer goes down?

i've inserted this middle man whosepurpose in life is to solve a problem. but i've introduced a new problem. a new leak has sprung in the hose. because now if the load balancerdies or breaks or misfunctions, now i lose access toall three of my servers. and before, i didn'thave this middleman. and so this is a new problem, arguably. we'll come back tohow we might fix that. david malan: that would be one approach.

yeah, and so this is going to be quitethe rat's hole we start to go down. but let's come back tothat in just a moment. what other problems have we created? >> so dan mentioned database before. and even if you're nottoo familiar technically, a database is just a server wherechanging data is typically stored, maybe an order someone has placed,your user profile, your name, your email address, things that mightbe inputted or changed over time. >> previously, my database was onthe same server as my web server.

because i just had oneweb hosting account. everything was all in the same place. where should i put my databasenow, on server 1, 2, or 3? >> audience: 4. >> david malan: 4, ok, allright, so let's go there. so i'm going to put mydatabase-- and let's start labeling these www, www, www. and i'm going to say,this is number four. and i'll say db for database.

ok, i like this. what line should ipresumably be drawing here? >> david malan: yeah, so the code,as we'll discuss tomorrow, presumably is the sameon all three servers. but it now needs to connect not to adatabase running locally but elsewhere. and that's fine. we can just give the database aname, as we have, or a number. and that all works fine. but what have we done?

we've horizontally scaled by havingthree servers instead of one, which is good. because now we can handlethree times as much load. >> and better yet, if one or twoof those servers goes down, my business can continue to operate. because i still have one, even if i'mkind of limping along performance-wise. but what new problem have iintroduced by moving the database to this separate serverinstead of on 1, 2, and 3? david malan: yeah, so now i haveanother single point of failure.

if my database dies, or needs tobe upgraded, or whatever, now sure, my website is online. and i can serve static,unchanging content. but i can't let users log in or changeanything or order anything, worse yet. because if 4 is offline,then 1, 2, and 3 really can't talk to it by definition. >> ok so yeah, and so this is whyi'm hesitating to draw this. so let's come back to that. i don't mean to keep pushing you off.

but the picture is veryquickly going to get stressful. because you need to starthaving two of everything. in fact, if you've ever seen themovie contact a few years ago with jodie foster-- no? >> ok, so for the two ofus who've seen contact, there's a relationship there where theyessentially bought two of something rather than one, albeitat twice the price. so it was sort of a playfulcomment in the movie. it's kind of related to this.

we could absolutely do that. and you've just costus twice as much money. but we'll come back to that. >> so we've solved this. this is like a slippery slope. i don't want to deal with havingto have a duplicate database. it's too much money. you know what? i want to have my databasejust like in version one

where each server hasits own local database. so i'm just going todraw db on each of these. >> so now each web serveris identical in so far as it has the same code, the samestatic assets, same pictures and text and so forth. and each has its own database. i fixed the single pointof failure problem. now i have a database. no matter which two or one of thesethings die, there's always one left.

but what new problem have i createdthat dan's solution avoided? >> david malan: yeah, ihave to sync them, right? because either i need to syncwho's going where-- in other words, if alice visits mysite, and she happened to get randomly or round robinedor whatever, to server number one, thereafter i have to alwayssend her to server 1. because if i send herto server 2, it's going to look like she doesn't exist there. >> i'm not going to have her order history.

i'm not going to have her profile there. and that just feels likeit's inviting problems. and when bob visits, ihave to send him always to the same server, 2, or whicheverone, and charlie to a third one, and consistently. this isn't unreasonable, though. this is calledpartitioning your database. and in fact this was whatfacebook did early on. >> if you followed the history offacebook, it started here at campus

as www.thefacebook.com. then it evolved once mark startedspreading into other campuses to be harvard.thefacebook.com andmit.thefacebook.com, and probably bu.thefacebook.com, and the like. and that was becauseearly on, i don't think you could have friends across campuses. but that's fine. because anyone from harvardgot sent to this server. anyone from bu got sent to this server.

anyone from mit got sentto this server-- in theory. i don't quite know all theunderlying implementation details. but he presumably partitioned people bytheir campus, where their network was. >> so that's good up until the pointwhere you need two servers for harvard, or three servers for harvard. and then that simplicitykind of breaks down. but that's a reasonable approach. let's always send aliceto the same place, always send bob to the same place.

but what happens if alice'sserver goes offline? bob and charlie can still buythings and log into the site. but alice can't. so you've lost a thirdof your user base. maybe that's better than 100%? but maybe it'd be nice if we couldstill support 100% of our users even when a third of ourservers goes offline. >> so we could sync what? not the users, per se, but thedatabase across all these servers.

so now we kind of need somekind of interconnection here so that the servers themselvescan sync-- not unreasonable. and in fact, this technology exists. in the world of databases, there'sthe notion of master-slave databases, or primary-secondary,where among the features is not only to store dataand respond with data, but also just to constantlysync with each other. so any time you write or savesomething to this database, it immediately gets "replicated"to the other databases as well.

>> and any time you read from it,it doesn't matter where you are. because if in theorythey've all synced, you're going to get the same view of the data. so this sounds perfect. there's got to be a catch. what might the catch be? >> david malan: yeah, so three timesas much stuff could go wrong. that's a reality. it might all be the same in spirit.

but someone needs to configure these. there's a higher probability thatsomething's going to go wrong. just combinatorially you havemore stuff prone to errors. what else is bad potentially? >> david malan: yeah, sosyncing can be bad. even as you might knowfrom backups and such, if you just are blindly makingbackups, what if something does go wrong on one database? you delete something you shouldn't.

you've immediately replicatedthat problem everywhere else. so victoria was talking-- backupswould be a good thing here. and so we'll get back to that. and to be clear, we're talkingnot about backups here per se. we're talking about true replicationor synchronization across servers. they're all live. they're not meant tobe used for backups. david malan: what's that? audience: higher--

david malan: higher cost. we've tripled the cost forsure, although at least in terms of the hardware. because a database isjust a piece of software. and a web server is a piece of software. it's probably free if we're usingany number of open source things. but if we are usingsomething like oracle, we're paying oracle more money perlicenses, or microsoft for access. there's got to be some other catch here.

it can't be this simple. >> so to your point, i think it waskareem, for geography earlier-- or no, roman, was it, for geography-- supposethat we're being smart about this, and we're putting one of our servers,and in turn our databases, in the us, and another in europe, another insouth america, another in africa, another in asia, anywhere wemight want around the world. we already know from our traceroutes that point a and point b, if they're farther apart,are going to take more time. >> and if some of you have usedtools, like facebook or twitter

or any of these sites these days thatare constantly changing because of user created data, sometimes if youhit reload or open the same page in another browser, you seedifferent versions, almost. you might see someone's statusupdate here but not here, and then you reload, and then itappears, and you reload again, and it disappears. in other words, keep aneye out for this, at least if you're using socialnetworking especially. >> again, just because thedata is changing so quickly,

sometimes servers do get out of sync. and maybe it's a super small window. but 200 milliseconds, maybeeven more than that-- it's going to take some non-zero amountof time for these databases to sync. and we're not justtalking about one request. if a company has thousands ofusers using it simultaneously, they might buffer. in other words, there mightbe a queue or a wait line before all of those databasequeries can get synchronized.

so maybe it's actually a few seconds. >> and indeed this is true i think evento this day with facebook, whereby when they synchronize fromeast coast to west coast, it has a non-trivialpropagation delay, so to speak, that you just kind of have to tolerate. and so it's not so mucha bug as it is a reality that your users might not seethe correct data for at least a few seconds. >> i see this on twitter a lotactually where sometimes i'll

tweet in one window, open another tothen see it to confirm that it indeed went up, and it's not there yet. and i have to kind of reload,reload, reload-- oh, there it is. and that's not because it wasn't saved. it just hasn't propagatedto other servers. >> so this trade-off, too-- do you reallywant to expose yourself to the risk that if the user goes to their orderhistory, it's not actually there yet? i see this on certain banks. it always annoys me when, well, for one,you can only go like six months back

in your bank statements in some banks,even though in theory they should be able to have everything online. they just take stuff offline sometimes. sometimes, too-- what website is it? there's one-- oh, it's godaddy, i think. godaddy, when you check outbuying a domain name or something, they'll often give youa link to your receipt. and if you click that link rightaway, it often doesn't work. it just says, dead end, nothing here.

>> and that's too because ofthese propagation delays. because for whatever reason, theyare taking a little bit of time to actually generate that. so this is sort of like you want topull your hair out at some point. because all you're trying todo is solve a simple problem. and we keep creating newproblems for ourselves. so let's see if wecan kind of undo this. >> it turns out that combiningdatabases on all of your web servers is not really best practice.

generally, what an engineerwould do, or systems architect, would be to have differenttiers of servers. and just for space's sake, i'lldraw their database up here. >> we might have database andserver number four here that does have connections toeach of these servers here. so this might be our frontend tier, as people would say. and this would be our back end tier. and that just means thatthese face the user. and the databases don't face the user.

no user can directlyaccess the database. >> so let's now maybe go downthe route victoria proposed. this is a single point of failure. that makes me uncomfortable. so what's perhaps themost obvious solution? david malan: sorry, say that again. david malan: non-production server. what do you mean? >> david malan: oh, ok, so backups.

ok, so we could do that, certainly. and actually this is very commonly done. this might be database number five. but that's onlyconnected to number four. and you might call it a hot spare. these two databases could be configuredto just constantly synchronize each other. and so if this machine dies, forwhatever stupid reason-- the hard drive dies, someone trips over thecord, some software is flawed

and the machine hangs or crashes--you could have a human literally unplug this one from the walland instead plug this one in. and then within, let's say, afew minutes, maybe half an hour, you're back online. >> it's not great, butit's also not horrible. and you don't have to worryabout any synchronization issues. because everything is already there. because you had a perfectbackup ready to go. >> you could be a littlefancier about this,

as some people often do, where youmight have database number four here, database number five here,that are talking to each other. but you also have thiskind of arrangement-- and it deliberatelylooks messy, because it is-- where all of thefront end servers can talk to all of the back end servers. and so if this database doesn'trespond, these front end servers have to have programmingcode in them that says, if you don't get aconnection to this database,

the primary immediately startstalking to the secondary. >> but this now pushes thecomplexity to the code. and now your developers, your softwaredevelopers, have to know about this. and you're kind of tying the code thatyou're writing to your actual back end implementation details,which makes it harder, especially in a biggercompany or a bigger website, where you don't necessarilywant the programmers to have to know how the databaseengineers are doing their jobs. you might want to keep those rolessort of functionally distinct so

that there's this layer ofabstraction between the two. >> so how might we fix this? well, we kind of solvedthis problem once before. why don't we put one ofthese things here where it talks in turn to number four andfive, all of the front end web servers talk to this middleman, and themiddleman in turn routes their data? in fact, what might be agood name for this thing? david malan: ok, database manager. but what might a term be thatwe could reuse for this device?

we're balancing. yeah, so actually, i'mnot being fair here. so a load balancer would imply thatwe're toggling back and forth here, which needn't actually be the case. so there's a few ways we could do this. >> if this is in fact a load balancer, thestory is exactly the same as before. some of the requests go to 4. some of them go to 5. and that's good.

because now we can handletwice as much throughput. but this connectionhere is super important. they have to stay constantlysynchronized and hopefully are not geographically too far apart sothat the synchronization is essentially instantaneous. otherwise we might have a problem. >> so that's not bad. but again, we'veintroduced a new problem. what problem have i just recreated?

single point of failure. so what's the solution to that? so as victoria's fond to spend money,we can take this guy out and do this. and i'm just going tomove here enough room. and it's going to be a little messy. i'm going to keep drawing lines. suppose that all ofthose lines go into both? >> a very common technique here would beto use a technique called heartbeat whereby each of these devices,left and right load balancers,

or whatever we want to call them,is constantly saying, i'm alive, i'm alive, i'm alive, i'm alive. one of them by defaultacts as the primary. so all traffic is being routed throughthe one on the left, for instance, by default, arbitrarily. >> but as soon as the guy on the rightdoesn't hear from the left guy anymore, the one on the right is programmedto automatically, for instance, take over the ip addressof the one on the left, and therefore become the primary, andmaybe send an email or a text message

to the humans to say, hey,the left primary is offline. i will become primary for now. so vice president becomespresident, so to speak. and someone has to go savethe president, if you want. because now we have a temporarysingle point of failure. >> so as complicated or stressful asthis might seem to start being, this is how you solve these problems. you do throw money at it. you throw hardware at it.

but unfortunately youadd complexity for it. but the result, ultimately, is thatyou have a much more, in theory, robust architecture. it's still not perfect. because even when we have-- we mightnot have a single point of failure. we now have dual points of failure. but if two things go wrong,which absolutely could, we're still going to be offline. >> and so very common in theindustry is to describe

your up time in terms of nines. and sort of the goalto aspire to is 99.999% of the time your site is online. or even better, add afew more nines to that. unfortunately, thesenines are very expensive. and let's actually do this out. so if i open up my big calculator again,365 days in a year, 24 hours in a day, 60 minutes in an hour, and60 seconds in a minute, that's how many seconds there arein a year if i did this correctly.

so if we times this by .99999, that'show much time we want to aspire to. so that means we should be upthis many seconds during the year. so if i now subtract theoriginal value, or rather this new value from thefirst-- 316 seconds, which of course is five minutes. >> so if your website or your company isclaiming "five nines," whereby you're up 99.99% of the time,that means you better have been smart enough and quickenough and flush enough with resources that your servers are only offlinefive minutes out of the year.

it's an expensive andhard thing to aspire to. >> so it's a trade off, too. 99.999% of the time is prettydarn hard and expensive. five minutes-- you can barely getto the server to physically replace something that's gone wrong. and that's why we start wiringthings together more complicated apriori so that the computerscan sort of fix themselves. yeah. david malan: the problem couldbe in any number of places.

and in fact-- david malan: absolutely, absolutely. and as the picture isgetting more complicated, it could be the web servers. it could be the power to the building. it could be something physical, likethe cables got frayed or kicked out. it could be the databaseisn't responding. it could be they updated their operatingsystem and something is hanging. so there are so many other moving parts.

and so a lot of the engineeringthat has to go behind this is really just trade offs, like howmuch time, how much money is it actually worth, and what are the threatsyou're really worried about? for instance, in thecourses i teach at harvard, we use a lot of cloud computing, whichwe'll start taking a look at now, in fact, where we useamazon web services. just because that's theone we started with. but there's ever more these daysfrom google and microsoft and others. and we consciously choose to put allof our courses' virtual machines,

as they're called, in the i thinkit's western virginia data center. most of our studentshappen to be from the us, though there are certainlysome internationally. >> but the reality is it's justsimpler and it's cheaper for us to put all of our eggsin the virginia basket, even though i know if somethinggoes wrong in virginia, as has occasionally happened-- likeif there's a hurricane or some weather event like that, if there's somepower grid issue or the like-- all of our courses' data might go offlinefor some number of minutes or hours

or even longer. >> but the amount of complexitythat would be required, and the amount of money that wouldbe required, to operate everything in parallel in europe or in californiajust doesn't make so much sense. so it's a rational tradeoff, but a painful one when you're actuallyhaving that downtime. >> well, let's transition right now tosome of the cloud-based solutions to some of these problems. everything we've beendiscussing thus far

is kind of problems that havebeen with us for some time, whether you have your ownservers in your company, whether you go to a co-locationplace like a data center and share space with someone else,or nowadays in the cloud. >> and what's nice aboutthe cloud is that all of these things i'mdrawing as physical objects can now be thought of assort of virtual objects in the cloud that aresimulated with software. in other words, the computers today,servers today, like the dell picture

i showed earlier, are so fast, haveso much ram, so much cpu, so much disk space, that people have writtensoftware to virtually partition one server up into the illusion of itbeing two servers, or 200 servers, so that each of us customershas the illusion of having not just an account on some webhost, but our own machine that we're renting from someone else. >> but it's a virtual machine inso far as on one dell server, it again might be partitioned up intotwo or 200 or more virtual machines, all of which give someone administrativeaccess, but in a way where none of us

knows or can access other virtualmachines on the same hardware. so to paint a picture in today's slides,i have this shot here from a website called docker. >> so this is a little moredetail than we actually need. but if you view this asyour infrastructure-- so just the hardware your own,your servers, the racks, the data center, and all of that-- you wouldtypically run a host operating system. so something like-- it could be windows. it wouldn't be mac os.

because that's not reallyenterprise these days. so it would be linux or solarisor unix or bsd or freebsd or any number of other operating systemsthat are either free or commercial. >> and then you run aprogram, special program, called a hypervisor, orvirtual machine monitor, vmm. and these are products, if you'refamiliar, like vmware or virtualbox or virtual pc or others. and what those programs do is exactlythat feature i described earlier. it creates the illusionthat one physical machine

can be multiple virtual machines. >> and so these colorful boxes up top ispainting a picture of the following. this hypervisor, thispiece of software, call it vmware, running on some otheroperating system, call it linux, is creating the illusion thatthis physical computer is actually one, two, three virtual computers. so i've now bought, as the owner ofthis hardware, one physical computer. and now i'm rentingit to three customers. >> and those three customers all thinkthey have a dedicated virtual machine.

and it's not bait and switch. it's more disclosure thatyou're using a virtual machine. but technologically, we allhave full administrative control over each of those guestoperating systems, which could be any number of operating systems. >> i can install anything i want. i can upgrade it as i want. and i don't even have to know orcare about the other operating systems on that computer,the other virtual machines,

unless the owner of all this graystuff is being a little greedy and is overselling his or her resources. >> so if you're taking onephysical machine and selling it to not 200 but 400customers, at some point we're going to trip into thosesame performance issues as before. because you only have a finiteamount of disk and ram and so forth. and a virtual machineis just a program that's pretending to be afull fledged computer. so you get what you pay for here.

>> so you'll find online you might pay areputable company maybe $100 a month for your own virtual machine, oryour own virtual private server, which is another term for it. or you might find some fly bynight where you pay $5.99 a month for your own virtual machine. but odds are you don't have nearlyas much performance available to you, because they've been overselling itso, than you would with the higher tier of service or the better vendor. >> so what does this actually mean for us?

so let me go to this. i'm going to go to aws.amazon.com. just because they havea nice menu of options. but these same lessons apply to awhole bunch of other cloud vendors. unfortunately, it's often moremarketing speak than anything. and this keeps changing. so you go to a website like this. and this really doesn'ttell you much of anything. >> and even i, as i look at this, don'treally know what any of these things

necessarily do until i dive in. but let's start on the left, compute. and i'm going to click this. and now amazon has frankly anoverwhelming number of services but amazon ec2 is perhaps the simplest. >> amazon ec2 will create for us exactlythe picture we saw a moment ago. it's how they make a lot oftheir money in the cloud. apparently netflix and othersare in the cloud with them. this is all typicallyfluffy marketing speak.

so what i want to do is go to pricing--or rather let's go to instances first just to paint a picture of this. >> so this will vary by vendor. and we don't need to get too deep intothe weeds here of how this all works. but the way amazon, for instance,rents you a virtual machine or a server in the cloud is they've gotthese sort of funny names, like t2.nano, which means small,or t2.large, which means big. each of them gives you eitherone or two virtual cpus. >> why is it a virtual cpu?

well, the physical machine mighthave 64 or more actual cpus. but again, through software,they create the illusion that that one machine can bedivvied up to multiple users. so we can think of this ashaving one intel cpu or two. cpu credits per hour-- i wouldhave to read the fine print as to what this actually means. it means how much of the machineyou can use per hour vis-a-vis other customers on that hardware. >> here's how much ram or memory youget-- either half a gigabyte, or 500

megabytes, or 1 gigabyte, or 2. and then the storage just refers towhat kind of disks they give you. there's different storagetechnologies that they offer. but more interesting than thisthen might be the pricing. >> so if you are the cto oran engineer who doesn't want to run a server in youroffice, for whatever reason, and it's way toocomplicated or expensive to buy servers and co-locate them andpay rent in some physical cage space somewhere-- you just want to sitat your laptop late at night,

type in your credit card information,and rent servers in the cloud-- well, we can do it here. i'm going to go down to-- linuxis a popular operating system. and let's just get a sense of things. whoops-- too big. >> so let's look at their tiniestvirtual machine, which seems to have, for our purposes, one cpuand 500 megabytes of ram. that's pretty small. but frankly, web servers don'tneed to do all that much.

you have better specs in your laptop. but you don't need thosespecs these days for things. you're going to pay $0.0065 per hour. >> so let's see. if there are 24 hours in a day, andwe're paying this much per hour, it will cost you $0.15 to rent thatparticular server in the cloud. and that's just for a day. if we do this 365-- $57 torent that particular server. so it sounds super cheap.

>> that's also super low performance. so we, for courses i teach here, tendto use i think t2.smalls or t2.mediums. and we might have a few hundredusers, a few thousand users, total. it's pretty modest. so let's see what this would cost. so if i do this cost times 24hours times 365, this one's $225. and for the coursesi teach, we generally run two of everything, forredundancy and also for performance. so we might spend, therefore,$500 for the servers

that we might need per year. >> now, if you need more performance--let's take a look at memory. we've talked about memory quite a bit. and if you do need morememory-- and 64 gigabytes is the number i kept mentioning--this is almost $1 per hour. and you can pretty quickly see wherethis goes-- so 24 hours times 365. so now it's $8,000 per yearfor a pretty decent server. >> so at some point, there'sthis inflection point where now we could spend $6,000probably and buy a machine like that

and amortize its cost over maybe two,three years, the life of the machine. but what might push you infavor or disfavor of renting a machine in the cloud like this? again, this is comparable, probably,to one of those dell servers we saw pictured a bit ago. >> david malan: yeah, that's a huge upside. because we're not buying themachine, we don't have to unbox it. we don't have to lift it. we don't have to plug it into our rack.

we don't have to plug it in. we don't have to paythe electrical bill. >> we don't have to turnthe air conditioning on. when a hard drive dies, we don't haveto drive in in the middle of the night to fix it. we don't have to set up monitoring. we don't have to-- the list goes onand on of all of the physical things you don't need to dobecause of "the cloud." >> and to be clear, cloud computingis this very overused term.

it really just means paying someoneelse to run servers for you, or renting space onsomeone else's servers. so the term "cloud computing" is new. the idea is decades old. so that's pretty compelling. >> and what more do you get? well, you also get the ability todo everything on a laptop at home. in other words, all of thepictures i was just drawing-- and it wasn't that long ago that eveni was crawling around on a server floor

plugging the cables in foreach of the lines that you see, and upgrading the operatingsystems, and changing drives around. there's a lot ofphysicality to all of that. >> but what's beautiful about virtualmachines, as the name kind of suggests, now there are web-basedinterfaces whereby if you want the equivalentof a line from this server to another, just type, type, type,click and drag, click submit, and voila, you have it wired up virtually. because it's all done in software.

and the reason it's donein software is again because we have so much ram and somuch cpu available to us these days, even though all ofthat stuff takes time, it is slower to run thingsin software than hardware, just as it's slower to use a mechanicaldevice like a hard drive than ram, something purely electronic. we have so many resourcesavailable to us. we humans are sort of invariantly slow. and so now the machines can doso much more per unit of time.

we have these abilitiesto do things virtually. >> and i will say for coursesi teach, for instance, here, we have about maybe a dozen orso total of virtual machines like that running at any giventime doing front end stuff, doing back end stuff. we have all of our storage. so any videos, including thingslike this that we're shooting, we end up putting into the cloud. amazon has services called amazon s3,their simple storage service, which

is just like disk space in the cloud. they have somethingcalled cloudfront, which is a cdn service, contentdelivery network service, which means they take all of your files andfor you automagically replicate it around the world. >> so they don't do it preemptively. but the first time someonein india requests your file, they'll potentially cache it locally. the first time in china, thefirst time in brazil that happens,

they'll start caching it locally. and you don't have to do any of that. and so it is so incrediblycompelling these days to move things into the cloud. because you have this ability literallyto not have humans doing nearly as much work. and you literally don't need as manyhumans doing these jobs anymore-- "ops," or operational roles, anymore. you really just needdevelopers and fewer engineers

who can just do things virtually. in fact, just to giveyou a sense of this, let me go to pricing forone other product here. let's see something like cdn s3. so this is essentially avirtual hard drive in the cloud. and if we scroll down to pricing--so it's $0.007 per gigabyte. and that's-- how do we do this? i think that's per month. >> so if that's per month-- or per day?

dan, is this per day? this is per month, ok. so if this is per month--sorry, it's the $0.03 per month. there's 12 months out of the year. so how much data mightyou store in the cloud? a gigabyte isn't huge, but idon't know, like 1 terabyte, so like 1,000 of those. that's not all that much. it's $368 to store a terabyteof data in amazon's cloud.

so what are some ofthe trade offs, then? it can't all be good. nothing we've talked about today issort of without a catch or a cost. so what's bad about movingeverything into the cloud? audience: security. david malan: ok, what do you mean? david malan: yeah, right. and do you really wantsome random engineers at amazon that you'll never meet havingphysical access to those computers,

and if they reallywanted, virtual access? and even though intheory software-- well, encryption can absolutelyprotect you against this. so if what you'restoring on your servers is encrypted-- less of a concern. >> but as soon as a human has physicalaccess to a machine, encryption aside, all bets are sort of off. you might know from yesteryearthat pcs especially, even if you had those thingscalled "bios passwords,"

were when your desktop booted up,you'd be prompted with a password that has nothing to do withwindows, you can typically just open the chassis of themachine, find tiny little pins, and use something calleda jumper and just connect those two wires for about a second,thereby completing a circuit. and that would eliminate the password. >> so when you have physical access to adevice, you can do things like that. you can remove the hard drive. you can gain access to it that way.

and so this is why, inthe case of dropbox, for instance, it's a littleworrisome that not only do they have the data, even though it'sencrypted, they also have the key. other worries? david malan: yeah, it's verytrue-- the googles, the apples, the microsofts of the world. and in fact, how long haveyou had your iphone for? yeah, give or take. you're among those whohas an iphone, right?

audience: yes. david malan: how longhave you had your iphone? david malan: ok, soapple literally knows where you've been every hour ofthe day for the last five years. david malan: which isa wonderful feature. david malan: yeah, buttrade off for sure. >> david malan: yeah, it's very easy to. david malan: other downsides? david malan: absolutely--technologically,

economically, it's pretty compelling tosort of gain these economies of scale and move everything intothe so-called cloud. but you probably do want togo with some of the biggest fish, the amazons, the googles, themicrosofts-- rackspace is pretty big-- and a few others, and notnecessarily fly by night folks for whom it's very easy to dothis kind of technique nowadays. and that's whom you canpay $5.99 per month to. but you'll certainlyget what you pay for. >> when you say [inaudible], that's whenthings like these five nines come up,

whereby even if technologicallywe can't really guarantee 99.999, we'll just build in some kindof penalty to the contract so that if that does happen, at leastthere's some cost to us, the vendor. and that's what you would typicallybe getting them to agree to. >> david malan: and theone sort of blessing is that even when we go down, forinstance, or even certain companies, the reality is amazon,for instance, has so many darn customers, well-known customers,operating out of certain data centers that when something really goes wrong,like acts of god and weather and such,

if there's any sort of silver lining,it's that you're in very good company. your website might be offline. but so is like half ofthe popular internet. and so it's arguably a littlemore palatable to your customers if it's more of an internetthing than an acme.com thing. but that's a bit of a cheat. >> so in terms of other things to look at,just so that we don't rule out others, if you go to microsoft azure, theyhave both linux and windows stuff that's comparable to amazon's.

if you go to google compute engine,they have something similar as well. and just to round outthese cloud offerings, i'll make mention of one other thing. this is a popular websitethat's representative of a class of technologies. the ones we just talkedabout, amazon, would be iaas, infrastructure as a service, where yousort of physical hardware as a service. there's saas. actually, let me jot these down.

>> iaas-- infrastructureas a service, saas, and paas, which areremarkably confusing acronyms that do describe threedifferent types of things. and the acronyms themselvesdon't really matter. this is all of the cloud stuffwe've just been talking about, the lower level stuff, thevirtualization of hardware and storage in the so-called cloud, whether it'samazon, microsoft, google, or other. >> software as a service--all of us kind of use this. if you use google appsfor gmail or calendaring,

any of these web-basedapplications that 10 years ago we would have double clicked icons onour desktop, software as a service is now really web application. and platform as aservice kind of depends. >> and one example i'll give you herein the context of cloud computing-- there's one company that's quitepopular these days, heroku. and they are a service,a platform, if you will, that runs on top ofamazon's infrastructure. and they just make it even easierfor developers and engineers

to get web-based applications online. >> it is a pain, initially, to useamazon web services and other things. because you actually haveto know and understand about databases and web servers andload balancers and all the stuff i just talked about. because all amazon has done is nothidden those design challenges. they've just virtualized themand move them into a browser, into software instead of hardware. >> but companies like heroku and otherpaas providers, platform as a service,

they use those barebone fundamentalsthat we just talked about, and they build easier touse software on top of it so that if you want to get a web-basedapplication online these days, you certainly have toknow how to program. you need to know java or python or phpor ruby or a bunch of other languages. >> but you also need a place to put it. and we talked earlier aboutgetting a web hosting company. that's sort of the like mid-2000sapproach to getting something online. nowadays you might instead pay someonelike heroku a few dollars a month.

and essentially, once you'vedone some initial configuration, to update your website, youjust type a command in a window. and whatever code you've writtenhere on your laptop immediately gets distributed to any numberof servers in the cloud. >> and heroku takes care ofall of the complexity. they figure all the databasestuff, all the load balancing, all of the headaches that we'vejust written on the board, and hide all of that for you. and in return, you justpay them a bit more.

so you have these infrastructures asa service, platforms as a service, and then software as a service. it's, again, thisabstraction or layering. >> any questions on the cloud orbuilding one's own infrastructure? all right, that was a lot. why don't we go ahead andtake our 15 minute break here. we'll come back with a few new conceptsand a bit of hands-on opportunity before the evening is over.