After our update last Thursday, we thought we were done with the competition. It was time to finish up the rest of our finals and enjoy one last weekend at Olin. It had been a good run, but we didn't meet our original goal of beating the lenstool method.
Today I got home for winter break, excited to catch my dad (hi dad!) up on everything that happend this semester. I was telling about about the competition, and went to show him the leaderboard - it had been updated since the competition ended on Sunday. The final results were now visible! Recall that the public leaderboard was only based on 25% of predictions - the private leaderboard (based on the other 75% of predictions) determines the final winner.
Geoff and I came in 96th out of 357 participants! This is way better than our public leaderboard rank of 186th place. Furthermore, our score of 0.89688 blew the lenstool's score of 1.05 out of the water. We beat our original goal!
Wednesday, December 19, 2012
Thursday, December 13, 2012
A (moderately embarassing) update
After finishing this blog last night, we were excited to share it with other participants in the competition. We posted a link on the competition's discussion forum. Today, we got the following response:
...and that, my readers, is a giant glaring "OOPS" on our part. We spent a lot of time trying to predict the number of halos, when in fact, this data was given to us. On the one hand, it means our final "sketchy" attempt was actually legitimate. On the other hand... we spent *a lot* of time working on a non-existent problem. On the third hand (we've got four of them between the both of us)...we've got another "lesson learned" to add to our list from the last post:
Read and re-read the rules of a competition before diving into it.
...and that, my readers, is a giant glaring "OOPS" on our part. We spent a lot of time trying to predict the number of halos, when in fact, this data was given to us. On the one hand, it means our final "sketchy" attempt was actually legitimate. On the other hand... we spent *a lot* of time working on a non-existent problem. On the third hand (we've got four of them between the both of us)...we've got another "lesson learned" to add to our list from the last post:
Read and re-read the rules of a competition before diving into it.
Wednesday, December 12, 2012
Conclusion
We didn't meet our original goal of beating the LENSTOOL metric, but realistically, this was more of a learning experience for us than a competition. By viewing it primarily as a learning experience, we can confidently say that it has been a resounding success. Below are some of the things we learned:
1. Machine learning is not a "one approach fits all" type of field. None of the techniques we learned during the digit recognition competition were helpful, and recognizing this took a couple of weeks. It was a disheartening realization at first, but recognizing forced us to think more creatively than we might have otherwise.
2. It's possible to devise an effective ML algorithm without lots of training data. We didn't figure out how to do this, but some people evidently did (as shown in the leaderboard screenshot below).
We're excited to find out about the approaches that successful competitors used after the competition ends; these might serve as inspiration to us for future ML endeavors.
3. It's helpful to have an advanced degree when solving a problem for which canonical machine learning algorithms are not applicable. Looking at the profiles of top competitors, we saw that most of them had advanced degrees in areas like computer science and data mining; one guy even had a PhD in particle physics.
4. Our best legitimate approach was the very first one, which combined two different benchmark methods. It was the simplest method by far - perhaps the old adage is true, that simpler is better.
5. Spiral learning is a legitimate educational philosophy! This project reminded us both of "Modeling and Simulation", a required course for first-year students at Olin - we took it 3 years ago in Fall 2009 (class website). In that class we were taught the governing equations for certain real world systems, and learned the programming/modeling skills necessary to explain and predict the behavior of those systems. This competition enabled us to tackle a similar problem, but to a much higher degree - this time, we figured out the governing equations ourselves! It was really satisfying to use the same approach that we used freshman year, this time with three years of experience and confidence behind each of us.
Overall, we didn't do as well as we'd hoped, but we definitely learned a lot from the competition. It was a great experience to work on a real-world problem, with data that had to be deciphered and with new concepts that had to be learned. We were able to attack a difficult problem in a field which we had no prior knowledge about, and we were able to do so in creative ways. More than anything, this competition has given us the confidence and desire to tackle more data science problems in the future.
1. Machine learning is not a "one approach fits all" type of field. None of the techniques we learned during the digit recognition competition were helpful, and recognizing this took a couple of weeks. It was a disheartening realization at first, but recognizing forced us to think more creatively than we might have otherwise.
2. It's possible to devise an effective ML algorithm without lots of training data. We didn't figure out how to do this, but some people evidently did (as shown in the leaderboard screenshot below).
We're excited to find out about the approaches that successful competitors used after the competition ends; these might serve as inspiration to us for future ML endeavors.
3. It's helpful to have an advanced degree when solving a problem for which canonical machine learning algorithms are not applicable. Looking at the profiles of top competitors, we saw that most of them had advanced degrees in areas like computer science and data mining; one guy even had a PhD in particle physics.
4. Our best legitimate approach was the very first one, which combined two different benchmark methods. It was the simplest method by far - perhaps the old adage is true, that simpler is better.
5. Spiral learning is a legitimate educational philosophy! This project reminded us both of "Modeling and Simulation", a required course for first-year students at Olin - we took it 3 years ago in Fall 2009 (class website). In that class we were taught the governing equations for certain real world systems, and learned the programming/modeling skills necessary to explain and predict the behavior of those systems. This competition enabled us to tackle a similar problem, but to a much higher degree - this time, we figured out the governing equations ourselves! It was really satisfying to use the same approach that we used freshman year, this time with three years of experience and confidence behind each of us.
Overall, we didn't do as well as we'd hoped, but we definitely learned a lot from the competition. It was a great experience to work on a real-world problem, with data that had to be deciphered and with new concepts that had to be learned. We were able to attack a difficult problem in a field which we had no prior knowledge about, and we were able to do so in creative ways. More than anything, this competition has given us the confidence and desire to tackle more data science problems in the future.
One last hurrah
After giving up, the lingering question was "how well could we have done, given more time to devise an accurate method for predicting the number of halos?" Neither of us are big fans of unanswered questions, so we took a semi-sketchy approach to answering this question...
The competition's best benchmark predictions were produced by the existing LENSTOOL software, and those predictions were available to all participants. We had noticed early on that in these predictions, skies 1-40 each have one halo, skies 41-80 each have two halos, and skies each 81-120 each have three halos. We used this information and our position method to determine the positions of these known quantities of halos.
This last try achieved a score of 1.08476. It did not beat the LENSTOOL method (our original goal), but it was damn close. How well could we have done with more time to devise an accurate method for predicting the number of halos? Pretty well, in our humble opinion.
The competition's best benchmark predictions were produced by the existing LENSTOOL software, and those predictions were available to all participants. We had noticed early on that in these predictions, skies 1-40 each have one halo, skies 41-80 each have two halos, and skies each 81-120 each have three halos. We used this information and our position method to determine the positions of these known quantities of halos.
This last try achieved a score of 1.08476. It did not beat the LENSTOOL method (our original goal), but it was damn close. How well could we have done with more time to devise an accurate method for predicting the number of halos? Pretty well, in our humble opinion.
A New Hope
At this point in the competition, we had a pretty good method for predicting the position of halos. However, we had no accurate way of predicting the number of halos.
We realized that what we needed was an algorithm that can tell us more about each predicted halo so that we can determine if each predicted halo is real or not. So we switched gears and tried a new position predicting algorithm -- one that came to be known as the "better subtraction" algorithm, since it was a revision of the "subtracting the effect of known halos" method
Here's the basic idea behind "better subtraction": it predicts the first position of the halo using the signal method. It then "subtracts away" the effect of the first halo on every galaxy. The resulting sky is a good approximation of what the sky would look like without the first halo. The cycle of predict-and-subtract is repeated 4 more times, for a total of 5 predicted halo positions. Afterwards, we predict the number of halos (n) that exist in the sky, and the first n (out of 5) halos stand as our prediction
An in-depth look at "better subtraction"
So what does it mean to "subtract away" the effect of the first halo? Well, we know that halos cause galaxies to be more elliptical. If we can determine how much the first halo makes a galaxy elliptical, then we can modify the ellipticity of the galaxies to "remove" this halo!
To figure out how much to subtract away, we first create a copy of the sky in which all galaxies are perfect circles. We then figure out how elliptical the galaxies would be in this sky if the first halo was inserted into it. We'll define the amount of ellipticity due to the first halo with the components e1_h1 and e2_h2 (For a reminder of what these two coordinates mean, go here). Here's some facts that we know:
There's just one problem: we don't know what m -- the relative strength of the halo -- is... We only know its location and the signal at its position in the sky.
However, we can get a really good guess for what m should by guessing and checking. Let's say the coordinates of the first halo are (x_h1, y_h1). If we remove the effects of the halo, then we can expect the signal at (x_h1, y_h1) to be 0.* If there's nothing at (x_h1, y_h1) that's affecting the ellipticity of the galaxies, then there should be no net orientation of galaxies about that point!
Alright, with all that information combined, we now have a new algorithm, which is written out in pseudocode below:
make an initial guess of m
calculate signal at (x_h1, y_h1)
while (signal at (x_h1, y_h1) != 0):
if the signal is > 0:
make m greater (halo is stronger)
if the signal is < 0:
make m smaller (halo is weaker)
for each galaxy in the sky
calculate (e1_h1, e2_h2) given m
subtract (e1_h1, e2_h2) from (e1, e2)
calculate signal at (x_h1, y_h1)
What this code is doing, in essence, is guessing what m is until the signal at x_h1, y_h1 is 0. Once we find m and "subtract away" the first halo, we can repeat the same process to predict where a second halo would be, and a third halo, etc. This algorithm turns out to be about as accurate as our last one at predicting halo positions. It can predict the positions of the first two halos with high accuracy, but is not so great at predicting the third. Given the position of halos, it scored a 1.08, which is about the same as our last algorithm. So not much of an improvement with predicting halo positions...
Here's what we can do with m: if m is a large number, we expect it's more likely that the predicted halo actually exists, because it is unlikely that such a strong source would be the product of randomness. If m is small, it is much less likely that the predicted halo actually exists (although it could just be a smaller halo).
Because the halos varied in strength, we chose to look at the ratio between m-values for the predicted halos. Let m1 be the strength of the first predicted halo, let m2 be the strength of the second predicted halo, etc. If the sky has only 1 halo, we expect the ratio of m2:m1 to be low, since m2 represents the strength of a non-existent halo. Furthermore, we expect the ratio of m3:m2 to be approximately 1, because they represent equally non-existent halos. Generally, in a sky with n halos, we predict that mn+1:mn will be much less than 1, and that mn+2:mn+1 will be approximately 1.
To see this idea in action, let's take a look at the two CDFs below. The CDFs show the ratios m2:m1 and m3:m2 for a bunch of skies that only contain one halo:
The m2:m1 ratios are all less than 1, and the m3:m2 ratios follow a more normal distribution, with a mean near 1. This is what we would expect given that there's only one halo in the sky. Science works.
Here's a pesudocode implementation of our method for predicting the number of halos in the sky:
n = 1
while True:
calculate m_n and m_n+1 (do this by predicting the position of the nth halo and the n+1th halo)
ratio_n = m_n+1 and m_n
if (ratio_n is approx 1)
guess that there are n-1 halos in the sky
break loop
else
n++
So at this point we had an algorithm for predicting the positions of halos, and an algorithm for predicting the number of halos as well! We combined both of them and submitted another guess to Kaggle...
... and got a score of 1.49. Worse than our first attempt :/ (Remember that our first attempt was just blindly slapping together two benchmark algorithms).
As we continued to focus on predicting the number of halos, we finally figured out a way to apply a traditional machine learning algorithm: k-nearest neighbors! This seemed like the best ML algorithm to use, because the training set wasn't big enough for a neural network or other classifiers. Using the ratios m2:m1, m3:m2, m4:m3, m5:m4 as "features", and our training set of 300 halos, we classified each sky as either a one halo, two halo, or three halo sky. Our score on the test data didn't improve at all.
At this point, we tried doing a bunch of optimizations to improve our score. Here are some other things we looked at
Looking back, we think there were two main reasons our algorithm for predicting the number of halos didn't work very well.
In summary, if we wanted to improve our overall accuracy, our best bet would be improving our position-predicting algorithm. However, by this point we were close to the end of the semester, and we didn't really have time to try any new methods. And thus ended our dark matter quest.
* In reality, the signal at that point won't be zero, because other halos in the sky will be affecting the ellipticity of the galaxies. However, this simplifying assumption proved to be fairly accurate.
To figure out how much to subtract away, we first create a copy of the sky in which all galaxies are perfect circles. We then figure out how elliptical the galaxies would be in this sky if the first halo was inserted into it. We'll define the amount of ellipticity due to the first halo with the components e1_h1 and e2_h2 (For a reminder of what these two coordinates mean, go here). Here's some facts that we know:
- The amount that each galaxy is affected is proportional to 1/r, where r is the euclidean distance between the galaxy and the halo.
- If each galaxy was naturally circular and there was only the one halo in the sky, than each galaxy would appear tangential to the halo.
- We know that the tangential component of ellipticity can be calculated by the following equation:
-(e1_h1 * cos(2*phi) + e2_h1 * sin(2*phi))
where phi is the angle between the halo and the galaxy with respect to the x-axis.
Based on these two facts, we can predict e1_h1 and e2_h2 are with the following equations:
e1_h1 = (m / r) * -(tan(2*theta) sin(2*phi)) + cos(2*phi))
e2_h1 = e1_h1 * tan(2 * theta)where theta is the angle of the galaxy's major axis with respect to the x-axis, and m is the relative strength of the halo.
There's just one problem: we don't know what m -- the relative strength of the halo -- is... We only know its location and the signal at its position in the sky.
However, we can get a really good guess for what m should by guessing and checking. Let's say the coordinates of the first halo are (x_h1, y_h1). If we remove the effects of the halo, then we can expect the signal at (x_h1, y_h1) to be 0.* If there's nothing at (x_h1, y_h1) that's affecting the ellipticity of the galaxies, then there should be no net orientation of galaxies about that point!
Alright, with all that information combined, we now have a new algorithm, which is written out in pseudocode below:
make an initial guess of m
calculate signal at (x_h1, y_h1)
while (signal at (x_h1, y_h1) != 0):
if the signal is > 0:
make m greater (halo is stronger)
if the signal is < 0:
make m smaller (halo is weaker)
for each galaxy in the sky
calculate (e1_h1, e2_h2) given m
subtract (e1_h1, e2_h2) from (e1, e2)
calculate signal at (x_h1, y_h1)
What this code is doing, in essence, is guessing what m is until the signal at x_h1, y_h1 is 0. Once we find m and "subtract away" the first halo, we can repeat the same process to predict where a second halo would be, and a third halo, etc. This algorithm turns out to be about as accurate as our last one at predicting halo positions. It can predict the positions of the first two halos with high accuracy, but is not so great at predicting the third. Given the position of halos, it scored a 1.08, which is about the same as our last algorithm. So not much of an improvement with predicting halo positions...
Using position to predict quantity
Here's the really cool thing about this algorithm -- for each predicted halo, we get a number -- m -- that represents how strong the halo is. This is a new measurement that we didn't have before! And this statistic provides us with useful data for predicting how many halos actually exist in the sky.Here's what we can do with m: if m is a large number, we expect it's more likely that the predicted halo actually exists, because it is unlikely that such a strong source would be the product of randomness. If m is small, it is much less likely that the predicted halo actually exists (although it could just be a smaller halo).
Because the halos varied in strength, we chose to look at the ratio between m-values for the predicted halos. Let m1 be the strength of the first predicted halo, let m2 be the strength of the second predicted halo, etc. If the sky has only 1 halo, we expect the ratio of m2:m1 to be low, since m2 represents the strength of a non-existent halo. Furthermore, we expect the ratio of m3:m2 to be approximately 1, because they represent equally non-existent halos. Generally, in a sky with n halos, we predict that mn+1:mn will be much less than 1, and that mn+2:mn+1 will be approximately 1.
To see this idea in action, let's take a look at the two CDFs below. The CDFs show the ratios m2:m1 and m3:m2 for a bunch of skies that only contain one halo:
Here's a pesudocode implementation of our method for predicting the number of halos in the sky:
n = 1
while True:
calculate m_n and m_n+1 (do this by predicting the position of the nth halo and the n+1th halo)
ratio_n = m_n+1 and m_n
if (ratio_n is approx 1)
guess that there are n-1 halos in the sky
break loop
else
n++
So at this point we had an algorithm for predicting the positions of halos, and an algorithm for predicting the number of halos as well! We combined both of them and submitted another guess to Kaggle...
... and got a score of 1.49. Worse than our first attempt :/ (Remember that our first attempt was just blindly slapping together two benchmark algorithms).
As we continued to focus on predicting the number of halos, we finally figured out a way to apply a traditional machine learning algorithm: k-nearest neighbors! This seemed like the best ML algorithm to use, because the training set wasn't big enough for a neural network or other classifiers. Using the ratios m2:m1, m3:m2, m4:m3, m5:m4 as "features", and our training set of 300 halos, we classified each sky as either a one halo, two halo, or three halo sky. Our score on the test data didn't improve at all.
At this point, we tried doing a bunch of optimizations to improve our score. Here are some other things we looked at
- Subtracting away only a portion of the galaxies, and not the whole thing. We optimized over the training data to determine how much we should subtract away.
- Using probabilistic ratios to inform our predictions. In the training set, and the test set, we knew that 1/3 of the skies had one halo, 1/3 had two, and 1/3 had 3. We incorporated this information into our algorithm.
- Detecting anomalies. When we predicted the position of halos, a number of them appeared to be on the edge of the sky. We thought that these predictions might be anomalies, and we threw many of them out.
After trying all of these methods, our score didn't improve at all. And that's when we decided to give up.
Looking back, we think there were two main reasons our algorithm for predicting the number of halos didn't work very well.
- Lack of training data. It's hard to machine learn with only 300 training samples.
- Shaky position-predicting algorithm. Our algorithm for predicting the number of halos uses data from the position-predicting algorithm. Our position-predicting algorithm -- "better subtraction" -- was highly accurate at predicting the positions of the first and second halos, but not nearly as accurate in predicting the third. Therefore, it was hard to tell whether the predicted third sky was an actual sky or not. We confirmed this by looking at skies with the same number of halos. When there was only one halo in the sky, our algorithm predicted the number of halos correctly about 80% of the time. However, for two or three halos, this accuracy dropped down close to 40% -- not much better than random guessing.
In summary, if we wanted to improve our overall accuracy, our best bet would be improving our position-predicting algorithm. However, by this point we were close to the end of the semester, and we didn't really have time to try any new methods. And thus ended our dark matter quest.
* In reality, the signal at that point won't be zero, because other halos in the sky will be affecting the ellipticity of the galaxies. However, this simplifying assumption proved to be fairly accurate.
The black hole where code goes to die
Over the next few weeks, we tried a lot of approaches that did not work so well. The purpose of each of these approaches was to determine either a) the number of halos, or b) the positions of the halos, but not both. Our goal was to find a solution to each of these problems independently, then combine them into a single sophisticated algorithm. Below is a list of a few techniques that failed us:
We wanted to apply the techniques we'd learned from Coursera, and this benchmark lent itself well to parameter optimization. We tried several grid sizes (from 5x5 to 30x30, in increments of 5), and evaluated each one on the training data. The 25x25 grid worked best, but not by a significant margin. We applied the gridded signal method to the test data using this parameter, and achieved a score of 2.52128 when we submitted the results to Kaggle.
This shockingly terrible score (using its default parameters, the benchmark had a achieved a 1.58) was an artifact of how Kaggle scores competitors: it only analyzes 25% of a solution while the competition in progress; it analyzes the other 75% after the competition is over. The purpose behind this is to stop people from gaming the system - understandable, but frustrating. Given that the test data only has 120 skies, the score is calculated based on 30 predictions. It's simply not all that informative.
2. Subtracting the effect of known halos. This approach was our most original, and most successful. So successful in fact, that the entire next post will be devoted to it. Our first iteration of this method, however, was not so successful. Here's the basic idea: we calculate the position of the first halo, based off of the signal measurement. To find the position of the second halo, we calculated a modified signal which ignores off the effects of the first halo.
Assume that there's a halo at (x1, y1) and we're trying to calculate the signal at (x2, y2) due to the galaxy G. (Recall that the total signal at a point is based off of the individual signal that each galaxy produces.) Then, let s_g1 be the signal due to G at (x1, y1), and let s_g2 be the signal due to G at (x2, y2). Since we know there's a halo at (x1, y1), we're pretty certain that s_g2 contains some of the affects of the first halo.
Let v1 be the distance vector between (x1, y1) and the galaxy, and let v2 be the distance vector between (x2, y2), as seen in the picture. We can calculate s_g2_g1 -- the amount of s_g2 that was influenced by s_g1 -- with the following formula:
This approach was actually fairly accurate at predicting the positions of all three halos. The drawback to it is that it didn't contain any information that quantified the effect of each halo. As a result, we weren't able to predict the number of halos in the sky at all (as we discuss in the next paragraphs). For our second iteration of this algorithm -- which we'll discuss in the next post -- we tried a different subtraction method which gave us more information about each predicted halo.
2. Relationship between signal strength of halos. Given that we've predicted the position of 1 halo, and we predict the position of a second halo, what is the relationship between their signals? Our hypothesis was that when a second halo actually exists, its signal would be along the same order of magnitude as the first halo. When the second halo does not actually exist, its signal would be at least one order of magnitude less than the first halo. In fact, no such relationship existed.
While this second approach was unsuccessful, it inspired a later approach that was moderately successful, in which we looked at a different measure that quantified the strength of halos.
Finding the positions of halos
1. Optimizing grid size in the gridded signal benchmark. Recall the gridded signal benchmark (provided by Kaggle) that contributed to our first submission in the competition. It works by dividing the sky up into a 15x15 grid, where each grid box is a 280x280 unit square. It calculates the signal at the center of the box based only on the galaxies contained by the box, and the box with the highest signal is deemed to be the one containing the halo.We wanted to apply the techniques we'd learned from Coursera, and this benchmark lent itself well to parameter optimization. We tried several grid sizes (from 5x5 to 30x30, in increments of 5), and evaluated each one on the training data. The 25x25 grid worked best, but not by a significant margin. We applied the gridded signal method to the test data using this parameter, and achieved a score of 2.52128 when we submitted the results to Kaggle.
This shockingly terrible score (using its default parameters, the benchmark had a achieved a 1.58) was an artifact of how Kaggle scores competitors: it only analyzes 25% of a solution while the competition in progress; it analyzes the other 75% after the competition is over. The purpose behind this is to stop people from gaming the system - understandable, but frustrating. Given that the test data only has 120 skies, the score is calculated based on 30 predictions. It's simply not all that informative.
2. Subtracting the effect of known halos. This approach was our most original, and most successful. So successful in fact, that the entire next post will be devoted to it. Our first iteration of this method, however, was not so successful. Here's the basic idea: we calculate the position of the first halo, based off of the signal measurement. To find the position of the second halo, we calculated a modified signal which ignores off the effects of the first halo.
Assume that there's a halo at (x1, y1) and we're trying to calculate the signal at (x2, y2) due to the galaxy G. (Recall that the total signal at a point is based off of the individual signal that each galaxy produces.) Then, let s_g1 be the signal due to G at (x1, y1), and let s_g2 be the signal due to G at (x2, y2). Since we know there's a halo at (x1, y1), we're pretty certain that s_g2 contains some of the affects of the first halo.
s_g2_g1 = dot(v1,v2) * s_g1where dot is the dot product. Therefore if we subtract s_g2_g1 away from s_g2, it's like we're subtracting away the effects of the first halo!
This approach was actually fairly accurate at predicting the positions of all three halos. The drawback to it is that it didn't contain any information that quantified the effect of each halo. As a result, we weren't able to predict the number of halos in the sky at all (as we discuss in the next paragraphs). For our second iteration of this algorithm -- which we'll discuss in the next post -- we tried a different subtraction method which gave us more information about each predicted halo.
Finding the number of halos
1. Geometric relationships between halos. We had several hypotheses about the geometric relationships between halos, all of which turned out to be false.- The distance between halos in a sky with 2 halos will be significantly less than the distances between halos in a sky with 3 halos. (Nope, these distances turned out to have virtually identical normal distributions)
- In skies with 3 halos, the triangle created by connecting the halos will have certain minimum area. (Again, a normal distribution -- didn't tell us anything)
2. Relationship between signal strength of halos. Given that we've predicted the position of 1 halo, and we predict the position of a second halo, what is the relationship between their signals? Our hypothesis was that when a second halo actually exists, its signal would be along the same order of magnitude as the first halo. When the second halo does not actually exist, its signal would be at least one order of magnitude less than the first halo. In fact, no such relationship existed.
While this second approach was unsuccessful, it inspired a later approach that was moderately successful, in which we looked at a different measure that quantified the strength of halos.
Tuesday, December 11, 2012
Signal: The Foundation of all our Algorithms
In the last post we referred to the signal metric, which "we really didn't understand." As subscribers to Olin College's philosophy of "do-learn," the next step was understanding the signal metric. It played a crucial role in the rest of the algorithms we developed, so this post will be devoted to an in-depth explanation of what it is.
In particular, we looked at a number known as the signal at every point in each sky. The signal is a measurement of how the ellipticities of each galaxies align with each other at a given point in a sky. Here's the basic idea behind it: all galaxies are elliptical to some extent. If the sky is free of dark matter, we'd expect that their major axes would all be pointing in random directions, as shown in Figure 1.
The signal is a unit-less measurement of how tangential all the galaxies are to a given point in the sky. It is an average of the tangential ellipticity of each galaxy relative to that given point. The ellipticity of a point (x,y) tangential to (x' y') is as follows (equations from here)
In pseudocode, this algorithm is as follows.
for each x point:
for each y point:
sig = average([e_tangential for every galaxy in this sky])
if sig > max_sig:
predicted_x = this x
predicted_y = this y
max_sig = sig
add this signal to a matrix of (X, Y, Signal) #used for plotting
predicted_halo = (predicted_x, predicted_y)
To gain a better intuition about signals, take another look at Figures 1 and 2. In Figure 2, the point at the center of the figure has a large, positive signal. In Figure 1, there is no net alignment of the galaxies. Therefore the point at the center of the galaxy would have a signal number near 0.
If it appears that galaxies are aligning themselves away from a point (as they are in Figure 3, then it is very unlikely that there's a halo at that point. The center of this figure would have a negative signal number. If a sky has a small area that looked like Figure 3, it is likely there is a halo(s) elsewhere in the sky stretching the galaxies beyond their natural ellipticity.
In particular, we looked at a number known as the signal at every point in each sky. The signal is a measurement of how the ellipticities of each galaxies align with each other at a given point in a sky. Here's the basic idea behind it: all galaxies are elliptical to some extent. If the sky is free of dark matter, we'd expect that their major axes would all be pointing in random directions, as shown in Figure 1.
 |
| Figure 2: Elliptical galaxies in a sky with dark matter |
 |
| Figure 1: Elliptical galaxies in a sky with no dark matter |
However, in the presence of dark matter, the major axis of each galaxy will have a tendency to be tangential to the halo, as shown in Figure 2.
The signal is a unit-less measurement of how tangential all the galaxies are to a given point in the sky. It is an average of the tangential ellipticity of each galaxy relative to that given point. The ellipticity of a point (x,y) tangential to (x' y') is as follows (equations from here)
In pseudocode, this algorithm is as follows.
for each x point:
for each y point:
sig = average([e_tangential for every galaxy in this sky])
if sig > max_sig:
predicted_x = this x
predicted_y = this y
max_sig = sig
add this signal to a matrix of (X, Y, Signal) #used for plotting
predicted_halo = (predicted_x, predicted_y)
To gain a better intuition about signals, take another look at Figures 1 and 2. In Figure 2, the point at the center of the figure has a large, positive signal. In Figure 1, there is no net alignment of the galaxies. Therefore the point at the center of the galaxy would have a signal number near 0.
 |
| Figure 3: Elliptical galaxies in a sky with no halos (again) |
If it appears that galaxies are aligning themselves away from a point (as they are in Figure 3, then it is very unlikely that there's a halo at that point. The center of this figure would have a negative signal number. If a sky has a small area that looked like Figure 3, it is likely there is a halo(s) elsewhere in the sky stretching the galaxies beyond their natural ellipticity.
The larger the signal is at a point, the higher the probability is that a halo exists there. If we calculate the signal at every point in a sky, we get a pretty good estimate of where one of the halos is located. For efficiency reasons, we calculated the signal at every 100 points, not at every single point. Figure 4 contains an example sky plotted on the same grid as its signal.
 |
| Figure 4: Signal for a sky with one halo. |
 |
| Figure 5: Signal map of a sky with 3 halos |
This sky only has one halo, marked by the star. The galaxies are represented by the black ellipses on the figure, and the color map represents the signal. As you can see, the strongest signal occurs right where the halo is. So this measure seems to work very well!
Here's the problem: we don't always only have one halo in the sky. A lot of times there's two or three halos. The signal metric doesn't account for this, since it chooses the single point with the highest signal. Take a look at Figure 5 - it clearly indicates the location of one halo, but without the blue and pink stars, it would tell you nothing about the other two halos' existence. Almost all the skies we examined looked like this, regardless of how many halos were actually in the sky. Since one halo is generally larger than the others, it overshadows the signal at the other halos.
We hoped to expand this method to work for predicting more than 1 halo. In addition we needed a solid way of predicting how many halos are in a given sky. We had a few ideas, but some turned out to be more successful than others...
Subscribe to:
Posts (Atom)