Present

 

Sree Balan (last year was at UCL, now Cambridge)

Donnacha Kirk (UCL)

Dugan Witherick (UCL)

Antony Lewis (Cambridge)

Sarah Bridle (UCL)

Ofer Lahav (UCL)

Tom Kitching (Oxford)

Laura Watson (UCL)

Lisa Voigt (UCL)

Stefan Harmeling (Tuebingen)

Bernhard Schoelkopf (Tuebingen)

Michael Hirsch (Tuebingen)

Mandeep Gill (OSU)

Stephane Paulin-Henriksson (Saclay)

Adam Amara (Zurich)

Catherine Heymans (Edinburgh)

Richard Massey (Edinburgh)

Alan Heavens (Edinburgh)

Marina Shmakova (SLAC)

Anais Rassat (Saclay)

Sandrine Pires (Saclay)

Frederic Courbin (EPFL)

Guldariya Nurbaeva (EPFL)

 

 

2pm Donnacha Kirk: Introduction to the GREAT08 Challenge for newcomers

 

http://www.great08challenge.info/Mid_Challenge_Workshop/DK_slides_UCL_05_01_09.ppt

 

A video small mp4 (15MB) or larger mp4 (232MB) .

 

Sarah: Perhaps useful to add a few comments on where Q=1000 comes from?

 

Adam: Lots of surveys starting at the moment including Dark Energy Survey, PanSTARRS, LSST, satellites to be launched. These surveys are going to observe a large fraction of the whole sky outside our galaxy. We want to measure the dark energy equation of state w to an accuracy of 1 per cent using the future surveys including LSST and satellites. To do this we need Q=1000. (See Amara & Refregier 2006.)

 

Ofer: Does the relation between error on w and Q take into account other types of error due to other systematics, or are you freezing those?

 

Adam: Yes, there will be other contributions to the error, and the effect of all of these must combine such that Q=1000. For this challenge the effect is coming from the shape error.

 

Tom: The current shape measurement methods have a Q value which is sufficient for the current surveys. We can maybe live with Q~100 for the next 5 years.

 

Sarah: Simultaneously the simulated images will be more realistic.

 

 

2.30pm Feedback/questions/comments from GREAT08 participants

  

Antony Lewis (CLT)

 

http://www.great08challenge.info/Mid_Challenge_Workshop/AntonyGreat08.ppt

 

A video small mp4 (5.1MB)

 

CLT stands for central limit theorem

 

The problem here is we don't know anything about the galaxies. I'm trying to do something which is non-optimal but doesn't depend on the properties of the galaxies.

 

Only have to fit the radial profile after stacking. Using a spline for the radial profile. This is based on a paper by Konrad Kuijken in 1999 (KK99).

 

You only have to know the galaxies have the same shear. The psf could vary, in which case would need the average psf. There are some more detailed notes linked here: http://great08challenge.pbwiki.com/CLT_KK99

 

There is some issue about centroiding the galaxies correctly so that when they are stacked they are all lined up, and don't add extra smearing.

 

See slides for a list of feedback for GREAT08, and a wishlist.

 

May also put a video of this online if it recorded OK.

 

Shape noise

 

Sarah: You mentioned shape noise being a problem. You may be interested to know that we made the images in a way that should remove shape noise, to first order. Just to define shape noise: this is the noise due to the fact that galaxies are not intrinsically circular. So if you only had a single galaxy in your survey, you couldn't know whether the galaxy was originally that shape, with no shear applied, or whether the galaxy was originally circular, and all the apparent

ellipticity comes from the shear. If you have more galaxies in your survey then this will beat down the shape noise, but it will still be there e.g. if you happen to have more galaxies aligned along the x axis than along the y axis then you will overestimate g_1 slightly. There is a way of removing this from

simulation tests, by putting matched galaxy pairs in the simulation. So for every galaxy you put into the image, there is another galaxy which is the same but rotated by 90 degrees. If these galaxies are combined together with the same weight then the shape noise cancels.

 

Tom: See Massey et al 2007 STEP2. It cancels out to first order at least.

 

 

Feasibility of challenge

 

Antony: how do you know Q=1000 is possible?

 

Sarah: uncertainty on shear for one galaxy is about 0.2 (or better), and Q=1000 corresponds to an accuracy of 0.0003 on the shear measurements. We average together your results so that enough galaxies are included so that 0.2/sqrt(n) is sufficiently less than 0.0003, so Q=1000 is possible given random errors of ~0.2 per galaxy.

As a check on our simulations we also generated very low noise images and used the unweighted quadrupole moments method to get the shear (essentially equations B.3, B.4, B.5, B.9 of the handbook, removing the PSF by subtracting off the quadrupole moments for the stars from the quadrupole moments of the

convolved image), and we indeed recovered the shear to a much higher precision than required for this challenge.

So the "only" extra feature of the simulations above this is the photon noise that dominates the error budget (given that the shape noise mostly cancels).

 

Stacking methods

 

Adam: Stacking might not be practical with real images. Could you go down to smaller stacks than 10000 instead of 50?

 

Antony: May be possible to generalise. To use the CLT to get full pixel covariance then need roughly at least many galaxies as pixels.

 

Sarah: In fact, Tim Schrabback did come up with the idea of stacking as a way of 'cheating' during early discussions of GREAT08. But then we remembered that early methods of measuring shear did actually just find the mean shear in circles of a given radius (e.g. containing tens of galaxies). This "top hat

variance" averages the mean shear over all positions of the circle to get a result for a given size of circle. Then plot

this result as a function of circle radius. This is simply related to the power spectrum or correlation function

statistics that we now usually talk about.

 

Sarah: Defer discussion of wish list to GREAT08 future and GREAT09.

 

 

Tuebingen (Gauss)

 

Stefan: We were quite surprised with our Q value for the LowNoise method. We got about 12 for the RealNoise with the nan fixed. We are using the Known data to set some parameters of the Blind. We are not stacking - we average ellipticity measurements from each galaxy separately to get the final shear.

 

Tom: Is it model fitting?

 

Stefan: We can process 10000 galaxies in 8 minutes. We fit the parameters of a parametric model to the images.

 

Sarah: You asked an interesting question which we still need to get back to you about:

"Is Equation (B.12) in the handbook an approximation?".

I have so far said "I believe that this was originally derived for objects with elliptical isophotes, and is exact in this case. I am not sure whether it is generally true, although I suspect is is (hence this is not explicitly stated in the handbook). I would like to try deriving it using eqns B.8 and B.9."

 

Action Sarah: Derive it, and/or email Mike Jarvis to see if he knows.

 

 

EPFL/Lausanne

 

Frederic: We are trying to collect the different expertise in different groups in image processing labs. We have things with wavelets and denoising, image deconvolution, neural networks. We are not expecting anything this year. For now we downloaded the lownoise stuff. We may submit something by April, but we expect to do better on the next GREAT Challenge. We have more methods for building psf, undersampled psfs etc.

 

Tom: If people are working on methods they think are going to take a long time to develop then please don't give up because

there will be a continuing suite of simulations.

 

 

Stephane Paulin-Henrikkson

 

Stephane: I'm analysing the data. I do a very simple Sersic fit in 2d. I take into account the pixelisation, but not the non-Gaussianity of the probability distribution. It is very slow to run. It takes maybe 5 seconds per galaxy. So even if I parallelise it I will not be able to analyse the whole data set for the end of April. It is also very unstable because when you perform the minimisation it depends on the initial values you use, the seed. The optimal thing would be to change from IDL to

C, so I could use a better minimiser, e.g. minuit.

 

Adam: Would it help to have approximate estimates of the shear e.g. from KSB, as Antony was suggesting?

 

Tom: KSB is very simple quadrupole moment fitting -- should be quick to write code to do this, then get that seed to run.

 

 

Laura Watson, Filipe Abdalla

 

Laura: I am doing this as a final year undergraduate project with Filipe Abdalla. I am focussing on the LowNoise_Known images first. So far I've extracted the pixel data and put it through a neural network.

 

 

Alan Heavens

 

Using a general optimisation code called MOPED. It finds principle components very fast. Applied to galaxy spectra already. Uses the same galaxy model as lensfit.

 

Sree: is MOPED public?

 

Sarah: Is patented!

 

Get in touch with Alan for more info.

 

What galaxy model is used for lensfit?

 

Tom: Lensfit uses a exp + devauc where the two are aligned. Could imagine extending the parameter set to where these are not aligned. Or could fit a Sersic.

 

 

Chris Roat

 

Sarah: Chris used to work on shear estimation and since left for a real job, e.g. he got results on the first part of STEP2. Now working on this in spare time. Has downloaded all the data (and sent useful feedback about problems with mirrors).

 

 

Is there anything you want that isn't there?

 

The code

 

See talk by Antony: Would like the code used for the simulations.

 

Sree: The code is written in c++. There will be a pdf document to go on the web and another will be a docbook. This documentation will be ready in the next few weeks.

 

Sarah: The question is when to make this public - now or later e.g. after the challenge deadline.

 

Catherine: What's the reason to make it public now?

 

Antony: There may be questions about the accuracy of the simulations, e.g. whether enough angles were used etc.

 

Tom: We could make a general version of the code public. But if we made the actual GREAT08 simulation code public now then we'd be giving away some of the key assumptions that were used in GREAT08 i.e. the model assumed for the galaxies.

 

Sarah: One problem with making the simulation details public is that people could make a load of simulations, make a load of fudge factors to fit the simulations and then win the challenge that way, which is not the method we want to encourage.

 

Lisa: We don't have to make the catalogues public with the code, so people won't be able to make matching simulations, and then cheat as you describe.

 

Stefan: Happy without the code.

 

Adam: If we change things too much in the middle of the challenge then it is a bit messy.

 

Sarah: OK, lets bear all this in mind for GREAT09 then.

 

 

More feedback than just Q?

 

See talk by Antony: Would like more info on the type of errors contributing to the large Q e.g. calibration error.

 

Sarah: We are worried that then people could fudge their method to win (e.g. the calibration error is the factor by which the shears are wrong, so if we gave this number people could just multiply all their shears up accordingly and get a much better Q, without really improving their method).

 

Antony: I could imagine that some methods might go wrong more if mod shear > 0.02, and it could be useful to isolate this.

 

Sarah, others: Yes, we agree that just providing Q is a bit artificial, and not so interesting as the full breakdown. We hope that everyone will be prepared to join together after the GREAT08 Challenge deadline to contribute to a paper analysing the different regimes in which each method does best. For it to be a real competition we have this Q, but what we're all really interested in is the details. We are looking forward to discussing this after the deadline.

 

 

Final conference location?

 

cosmostats09, Zurich 26 - 31 July

 

KDD June Sun 28 - Wed July 1, 2009; Workshop proposals due: Jan 19, 2009

 

a random cosmology meeting = ?

 

other cs meetings?

 

separate meeting? - RAS, UCL, CL?, JPL?, exotic location?

 

NB potential clashes with Aspen June 14 - July 5

 

Cosmostat might have to be a Saturday, in which case we should do KDD.

 

 

Prize?

 

In addition to the expenses paid trips to the final meeting, should we throw in e.g. a Macbook air?

 

Stefan: telescope?

 

Adam: I like this

 

Tom: How about an expenses paid trip to Hawaii to see a telescope?

 

Richard: telescope makes a nice picture for the press release

 

Marina: How would people divide a telescope? Perhaps should have prizes for team members too?

 

Sarah: Thanks v much for your feedback. This is to be finalised shortly.

 

 

IEEE review

 

Sarah: Statistical problems connected to astrophysics. Call for proposals for a paper for an issue. Should be addressed at people from engineering. Could be a good place to advertise what we're doing.

 

Action Marina: To forward the email to the GREAT08 Team for discussion.

 

 

How can we advertise it better?

 

Stefan: In machine learing it is easy to get carried away working on theory. Finding nice applications is always an issue for us. Astronomy is a great application for us. It looks like there are lots of statistics problems to come.

 

Adam: So one appeal is the potential for growth?

 

Stefan: Yes, getting in to the problems astronomers are interested in.

 

Stefan: The question posed is not a classical machine learning  problem, so it doesn't fit directly into the existing work. People might be interested and then decide it is not e.g. a classification problem.

 

 

4pm GREAT09

 

See presentation by Tom Kitching: http://www.great08challenge.info/Mid_Challenge_Workshop/GREAT09-1.pdf

 

A video small mp4 (21MB) 

 

Stefan: could have different tracks - classification as a different track, then another track etc. Some people would get interested in the classification problem.

 

Adam: What's the maximum number of tracks before this gets out of hand?

 

Tom: Could have 3 tiers:

* source identification and classification

* shear measurement

* psf interpolation

 

Stefan: 3 sounds OK.

 

Donnacha: Why make it more complicated if people don't get Q=1000 in GREAT08?

 

Sarah: useful to still have the old images, to test new pipelines on

 

Lisa: Useful if one strand is a continuation of GREAT08, so people can run the same codes again?

 

Adam: Lausanne group is interested in reconstructing the PSF accurately enough.

 

Alan: There is a question of whether this should be within a weak lensing challenge, since classification and psf

interpolation are important problems in their own right.

 

Adam: Could focus it on lensing

 

More fedback from Antony to be carried through to GREAT09 planning:

* Would find it useful to have a starting point e.g. the KSB results available on the website.

* Would like more Known images to beat down shape noise.

 

Conclusion: Will have telecons on planning GREAT09. Email list will be based on GREAT08 Team. Get in touch with Tom if you'd

like to be involved in GREAT09.