This is a little exploration into something I was thinking about today.
My hypothesis is this:
Over time, the influenial members of a group will become dominant, such that the overall characteristics of the group will take on the properties of the influencers.
A little Mathematical Example To Set It Up:
If you graph a set of random data points it looks like a bumpy random graph.
If you have a bunch of random sets, and sum up the values at each point, you don’t get a random graph anymore. You get a uniform graph.
It’s like “White Noise”. You have a thousand signals all mashing up against each other - and the total sum of all the signals is a uniformly bland mildly randomly fluctuating signal we call white noise.
Now - let’s say you took the same system but introduced signal interference.
Let’s say some signals were “influencers” and others were “influenceable”. The influencers change nearby signals to be more like themselves. They don’t change themselves. The influenceable are dynamic, strongly picking up the properties of their neighbor signals, and weakly changing nearby signals to be more like themselves.
Now, in this system, initially, you’ll get a uniform signal, as you did before.
Again, my hypothesis is this:
Over time, the influencers will become dominant, such that the overall signal will take on the properties of the sum of the influencers.
What are the implications?
If there are lots of random influencers, you’ll get a uniform signal like before.
But if the number of influencers is small, or if the influencers tend to all have similar characteristics, you’ll get a signal that is driven by the influencers.
The original problem I wanted to solve was this:
Can I build a simulation to figure out how much of an impact an influential member of a group had compared to those who prefer to follow?
So I’m going to build something like this:
Take a 100x100 table of data, with each row being a “member” and each column being “some thing that they believe in to a certain extent”. The value of a particular node (x,y) tells us that x person feels F(x,y) about issue y.
Then I’ll put in a correlation of influence. Some people will be big influencers, others will tend to gravitate towards the ideas of others.
Each person x will have a list of “neighbors” that they directly affect. So, an influencer with a lot of neighbors should theoretically cause their values to become much more prominent in the overall system.
The question I want to find out is – how much of the overall signal over time is driven by the top X% of influencers? None? Lots?
How about if influencers operate in a heirarchy? Where some influencers influence other influencers.
Another question: How far away from the original signal can you get simply by selectively crafting the influencers? What if influencers can change the number of others that they influence over time?
hmmmmm..
One more: How simple of a system can you have before the effect is obvious?
Wednesday, July 25, 2007
Friday, June 01, 2007
Why Projects Always Take Longer Than You Think
When it comes to estimating tasks as a software engineer, most people usually estimate the time it will take for a series of tasks, total them up, and that’s about how long it will take. Some methods are most sophisticated than others, using average case and safe estimate cases, and totaling those up - but fundamentally, everyone is still working with averages.
The problem that most people run into is that they’re trying to intuitively fit a normal distribution on the length of a task, and pick the average time.
That’s all well and good. If you’re more or less right with your estimations of the average time, your total time will be accurate, right?
Well, maybe not.
The reason is that tasks don’t follow a true normal distribution.
But why?
In a true normal curve, you have a continuous number of possibilities for every option. But with real-world estimation, no task can take less than zero time. So, the probability distribution of a task is bounded by zero. The percentage of tasks that take zero time is zero, and grows from there. However, there is no upper bound on how long a task will take. The percentage of tasks that take infinite time drops to an infinitesimal amount over.
So, when you pick the average, you’re saying, half the time it’ll take longer, and half the time it’ll take less time, overall it’ll even out. But it won’t. The tasks that run short are bounded by 0, whereas the tasks that run long have no bound. Because of this the area of the curve before the 50/50 mark is less than the area after the curve.
You don’t want your average to be the 50/50 mark of when the task will be finished. You want your mark to be the 50/50 of the area, which takes into account the bounding problem.
This number will always be higher than the original.
Which is why you’re always late with your projects.
Or you’re just a slacker.
The problem that most people run into is that they’re trying to intuitively fit a normal distribution on the length of a task, and pick the average time.
That’s all well and good. If you’re more or less right with your estimations of the average time, your total time will be accurate, right?
Well, maybe not.
The reason is that tasks don’t follow a true normal distribution.
But why?
In a true normal curve, you have a continuous number of possibilities for every option. But with real-world estimation, no task can take less than zero time. So, the probability distribution of a task is bounded by zero. The percentage of tasks that take zero time is zero, and grows from there. However, there is no upper bound on how long a task will take. The percentage of tasks that take infinite time drops to an infinitesimal amount over.
So, when you pick the average, you’re saying, half the time it’ll take longer, and half the time it’ll take less time, overall it’ll even out. But it won’t. The tasks that run short are bounded by 0, whereas the tasks that run long have no bound. Because of this the area of the curve before the 50/50 mark is less than the area after the curve.
You don’t want your average to be the 50/50 mark of when the task will be finished. You want your mark to be the 50/50 of the area, which takes into account the bounding problem.
This number will always be higher than the original.
Which is why you’re always late with your projects.
Or you’re just a slacker.
Monday, May 21, 2007
Dynamic Electricity Billing
The nature of a power grid is such that the more power you have to generate at once, the more expensive every bit of power is.
There are a few good reasons for this:
1. As the demand rises, the power companies have to generate more power from older, dirtier, and less efficient power plants. These plants take produce less power for each dollar it takes to run them.
2. As a seasonal effect - fuel becomes more expensive when demand rises. Coal and gas are most susceptible to fuel prices. (Solar power is actually cheaper at peak times because peak times are hot sunny afternoons, when solar cells are most effective.)
Thus, it might cost $1,000 to produce the first megawatt of electricity, but $1,500 to generate a second megawatt.
When Demand Rises
The demand for electricity rises to its highest during the day between noon and six on summer days as homes and businesses are all running air conditioners. At night demand falls because outside temperatures drop and air conditioners don’t have to work as hard.
The other limiting factor of the power grid is that enough power plants need to be built to supply people at the maximum load time. This means that the power grid is frequently operating well below capacity. It also means that the limiting factor on how many power plants we need is strongly associate to the number of air conditioners running on the hottest summer days.
That’s not to say other things don’t require electricity too – but overall, its air conditioning that causes significant fluctuations in demand.
(Lighting doesn’t cause as much of a difference as you might imagine – mostly because businesses run their lights during the day while homes tend to run their lights at night.)
How Pricing Will Change
One change the electric suppliers have learned from the telephone industry is basing their rates on time of day. In the future, you’ll get billed a higher rate in the afternoon than at night. As a homeowner this is good news, since you can just push up your thermostat when you’re away from work, and overall your bills will drop. Businesses on the other hand will have to start thinking about more efficient electricity usage if they want to keep electrical costs down.
But maybe that’s a good thing.
As rates rise, demand will fall. A certain portion of people will turn up their thermostats knowing that it will cost them extra otherwise.
This will allow more old power plants to run at lower capacity on peak summer days. It’ll decrease the amount of coal and oil power companies need to buy to run their plants, which will lower the rate overall. In turn, the best plants will be providing the most power which will also moderate prices. This might even also quicken the speed for the oldest plants to get decommissioned in place of newer supplemental power plants, such as solar, which run best on peak days. Wind is also an option in many areas.
We’d be trading our most expensive and polluting plants for renewable energy. Or at least much cleaner coal plants.
Though I’d personally prefer a power plant that didn’t emit greenhouse gasses.
There are a few good reasons for this:
1. As the demand rises, the power companies have to generate more power from older, dirtier, and less efficient power plants. These plants take produce less power for each dollar it takes to run them.
2. As a seasonal effect - fuel becomes more expensive when demand rises. Coal and gas are most susceptible to fuel prices. (Solar power is actually cheaper at peak times because peak times are hot sunny afternoons, when solar cells are most effective.)
Thus, it might cost $1,000 to produce the first megawatt of electricity, but $1,500 to generate a second megawatt.
When Demand Rises
The demand for electricity rises to its highest during the day between noon and six on summer days as homes and businesses are all running air conditioners. At night demand falls because outside temperatures drop and air conditioners don’t have to work as hard.
The other limiting factor of the power grid is that enough power plants need to be built to supply people at the maximum load time. This means that the power grid is frequently operating well below capacity. It also means that the limiting factor on how many power plants we need is strongly associate to the number of air conditioners running on the hottest summer days.
That’s not to say other things don’t require electricity too – but overall, its air conditioning that causes significant fluctuations in demand.
(Lighting doesn’t cause as much of a difference as you might imagine – mostly because businesses run their lights during the day while homes tend to run their lights at night.)
How Pricing Will Change
One change the electric suppliers have learned from the telephone industry is basing their rates on time of day. In the future, you’ll get billed a higher rate in the afternoon than at night. As a homeowner this is good news, since you can just push up your thermostat when you’re away from work, and overall your bills will drop. Businesses on the other hand will have to start thinking about more efficient electricity usage if they want to keep electrical costs down.
But maybe that’s a good thing.
As rates rise, demand will fall. A certain portion of people will turn up their thermostats knowing that it will cost them extra otherwise.
This will allow more old power plants to run at lower capacity on peak summer days. It’ll decrease the amount of coal and oil power companies need to buy to run their plants, which will lower the rate overall. In turn, the best plants will be providing the most power which will also moderate prices. This might even also quicken the speed for the oldest plants to get decommissioned in place of newer supplemental power plants, such as solar, which run best on peak days. Wind is also an option in many areas.
We’d be trading our most expensive and polluting plants for renewable energy. Or at least much cleaner coal plants.
Though I’d personally prefer a power plant that didn’t emit greenhouse gasses.
Wednesday, May 09, 2007
Tracking Your Electricity Usage
A while back I thought about ways to improve energy usage - and taking away a concept I've learned from my profession - is that you can't do a good job improving something if you don't know exactly what's wrong with it.
That lead me to the idea of installing web-cameras with image recognition that constantly viewed my Electric, Gas, and Water meters. Then I'd have a program running on a computer reading from the web cameras, and graphing the data in real-time.
Thus, I'd always have real-time access to my energy usages, and it'd be able to really easly tell how much a hot shower cost me in real dollar terms -- or whatever other household activity I was doing.
The idea then would be to make this preatty cheap to build and install. Then improve the software to the point where it quickly points out likely problems in a user's energy usage.
I think a lot of people would improve their energy usage if they had much more frequent data than they get from their monthly bills.
Fortunately, the electric company where I live has an online interface that will give you daily usage in kW.
Though its not as good as hourly data might be, daily trends are very helpful because I can remember on particular days what I was doing that might have used more or less and get a feel for how much certain activities are costing.
It might even make me think twice about running the dryer for a few extra minutes next time.
That lead me to the idea of installing web-cameras with image recognition that constantly viewed my Electric, Gas, and Water meters. Then I'd have a program running on a computer reading from the web cameras, and graphing the data in real-time.
Thus, I'd always have real-time access to my energy usages, and it'd be able to really easly tell how much a hot shower cost me in real dollar terms -- or whatever other household activity I was doing.
The idea then would be to make this preatty cheap to build and install. Then improve the software to the point where it quickly points out likely problems in a user's energy usage.
I think a lot of people would improve their energy usage if they had much more frequent data than they get from their monthly bills.
Fortunately, the electric company where I live has an online interface that will give you daily usage in kW.
Though its not as good as hourly data might be, daily trends are very helpful because I can remember on particular days what I was doing that might have used more or less and get a feel for how much certain activities are costing.
It might even make me think twice about running the dryer for a few extra minutes next time.
Thursday, December 07, 2006
Thoughts from a Hack Game Developer
I’ve been reading about different ways that people go about designing games, and one of the concepts that really stuck with me was ‘If you build your game with X, you tend to get X.’
What I took away from it was, the way you design something will show up in the final product and your users will know. Be it a game, a book, software, or anything else creative.
This got me to thinking about how I design my games. I feel like I take the “whatever works”, sort of haphazard approach. I’ll be playing a game, watching a movie, or in the car, and I’ll get an idea and run with it. There’s nothing approaching a formal method I use to build games, nothing like rules engines, systems for diagramming decision trees, or anything sophisticated. The result is that I generally get a game with a cool concept, and some pretty good mechanics, but not a lot of cohesion between the pieces and certainly none of what the kids call ‘Polish’.
I’m basically the Amateur Hack archetype of a designer. The kind of guy a game publishing company would probably laugh out the door. I’m cool with that.
But still, sometimes I wonder – maybe if I used a little bit more sophisticated methods for building a game, then when people played the game, they’d say “this game is really tight, the rules are totally balanced and polished.” Yeah, whatev. I’m not trying to build polished games. I’m out to make fun games. You know, the kind of games that make me all psyched up. The kind of game that makes me want to make more games.
Some of my friends were really excited when my games came out. Not just because I made them. In fact, some people were pretty skeptical at first. They were just humoring a raving lunatic at first. But they were genuinely excited the first few times through the game.
And to me was saying that the premise was strong. Subsequent plays found parts of the games to become a mix of bland, predictable, complicated, or inconsistent. Over time the more annoying aspects started to stick out worse than the good things. -- The games weren’t polished. The players wanted the crap that I’d glossed over in the first iteration fixed.
That’s easier said than done.
I’ve found the things I do to create a new game are quite a bit removed from the tasks and methods used to tighten up a game.
At any rate, I need to figure out how to get better at the tightening up part of game development.
If I come up with something, I’ll let you guys know.
What I took away from it was, the way you design something will show up in the final product and your users will know. Be it a game, a book, software, or anything else creative.
This got me to thinking about how I design my games. I feel like I take the “whatever works”, sort of haphazard approach. I’ll be playing a game, watching a movie, or in the car, and I’ll get an idea and run with it. There’s nothing approaching a formal method I use to build games, nothing like rules engines, systems for diagramming decision trees, or anything sophisticated. The result is that I generally get a game with a cool concept, and some pretty good mechanics, but not a lot of cohesion between the pieces and certainly none of what the kids call ‘Polish’.
I’m basically the Amateur Hack archetype of a designer. The kind of guy a game publishing company would probably laugh out the door. I’m cool with that.
But still, sometimes I wonder – maybe if I used a little bit more sophisticated methods for building a game, then when people played the game, they’d say “this game is really tight, the rules are totally balanced and polished.” Yeah, whatev. I’m not trying to build polished games. I’m out to make fun games. You know, the kind of games that make me all psyched up. The kind of game that makes me want to make more games.
Some of my friends were really excited when my games came out. Not just because I made them. In fact, some people were pretty skeptical at first. They were just humoring a raving lunatic at first. But they were genuinely excited the first few times through the game.
And to me was saying that the premise was strong. Subsequent plays found parts of the games to become a mix of bland, predictable, complicated, or inconsistent. Over time the more annoying aspects started to stick out worse than the good things. -- The games weren’t polished. The players wanted the crap that I’d glossed over in the first iteration fixed.
That’s easier said than done.
I’ve found the things I do to create a new game are quite a bit removed from the tasks and methods used to tighten up a game.
At any rate, I need to figure out how to get better at the tightening up part of game development.
If I come up with something, I’ll let you guys know.
Thursday, November 09, 2006
Atlas / Ajax
I have to say that I'm totally psyched about the near future of ASP.Net development after attending a training last night.
Microsoft's framework for beefing up the user experience in .Net Web applications last night will really bring down the barrier to entry for companies to start exploiting the same tricks that have made Google the darlings of the internet community.
Of course, Google will undoubtedly keep pushing the envelope. But this framework will bring Google's type of web-apps to the mainstream business software development world.
Microsoft's framework for beefing up the user experience in .Net Web applications last night will really bring down the barrier to entry for companies to start exploiting the same tricks that have made Google the darlings of the internet community.
Of course, Google will undoubtedly keep pushing the envelope. But this framework will bring Google's type of web-apps to the mainstream business software development world.
Tuesday, September 19, 2006
Lacking Depth of Substance
There are so many new venues for media out there that didn’t exist a decade ago. You have You Tube, Deviant Art, YTMND, and several good music sites. All of these are great forms of expression for artists. But lately, I’ve gone back to reading books again.
I’ve been getting the feeling lately that our attention spans keep going down. There are some fantastic flash animations, images and video shorts out there – but it has taken up the attention space of telling stories with substance.
Maybe I’m just looking in the wrong places – but despite all the new forms of expression, it seems like we haven’t really seem much meaningful work. The field is dominated by a girth of fleetingly amusing or interesting snippets. Some things gratify my attention for a few minutes, only to be forgotten a few days later. Others allude to an enticing tip of an iceberg - only to find that the creator created nothing that exists beyond it. Just empty shells of promising ideas.
It isn’t that I don’t like many of the things out there: I just wish I saw some inkling of things with substance now and again.
But maybe that’s not the web’s strength. Maybe I’m just hoping to find depth in the wrong places.
I’ve been getting the feeling lately that our attention spans keep going down. There are some fantastic flash animations, images and video shorts out there – but it has taken up the attention space of telling stories with substance.
Maybe I’m just looking in the wrong places – but despite all the new forms of expression, it seems like we haven’t really seem much meaningful work. The field is dominated by a girth of fleetingly amusing or interesting snippets. Some things gratify my attention for a few minutes, only to be forgotten a few days later. Others allude to an enticing tip of an iceberg - only to find that the creator created nothing that exists beyond it. Just empty shells of promising ideas.
It isn’t that I don’t like many of the things out there: I just wish I saw some inkling of things with substance now and again.
But maybe that’s not the web’s strength. Maybe I’m just hoping to find depth in the wrong places.
Subscribe to:
Posts (Atom)