We decided to submit an update of Jeroen’s work on caching for model counters, based on his master thesis work. It was accepted, so last week Jeroen presented our work.
While I was naturally happy to see our work presented, I also very much enjoyed the rest of the program. It has been a while since I’ve seen so many interesting talks in a row. I particularly liked the contributions from Tuukka Korhonen and Lucas Bang.
Jeroen’s slides can be found here, and a recording of his presentation is also available online.
Thanks to the organisers! I am looking forward to attending next year’s edition!
I am very proud of my student Jeroen Rook, who defended his Master thesis today! The title of his presentation was: Caching in Model Counters: A Journey through Space and Time. With a very general audience, Jeroen had his work cut out for him to explain not only his work, but also the basics of propositional model counting.
After his presentation, Jeroen defended his work to me and his other advisers, Prof. Dr. Holger Hoos and Prof. Dr. Siegfried Nijssen. He handled our questions as well as those from the audience quite well, impressing us with his knowledge of the #P complexity class.
Actually, that is not true. I am an outstanding reviewer! Or so say the Association for the Advancement of Artificial Intelligence and the AAAI-21 Program Committee.
If I’m perfectly honest, I was quite insecure about reviewing for AAAI. It was my very first time as a PC member, and I was not quite sure that I was knowledgeable enough to be able to be sufficiently critical of the contents of a paper and to recognise flaws, especially those related to novelty and obscure theorems that have pages-long proofs.
My colleagues encouraged me, by reminding me that I had read and graded roughly 150 student papers/reports/theses already. For many of those, the feedback I provided led to new and improved versions, and in some cases even to scientific publications. All in all, my colleagues reckoned I was ready.
I then simply did the best I could. I did not receive any feedback from the AAAI-21 Program Committee, so I can only guess why they picked me among the thirteen outstanding PC members out of all those thousands. Just in case you are reading this looking for tips on how to be a good reviewer, here are my recommendations for reviewing anything, be it a scientific paper submitted for publication, or a student report for an (under)graduate course.
Be specific
All good feedback is actionable. For example: instead of saying “Section 4 is unclear”, say “I would find it easier to understand Section 4 if concept X was formally defined.” Note that this also reflects the fact that your feedback is your opinion, not fact. It also indicates what made Section 4 unclear to you, and what would improve Section 4’s clarity for you.
If authors have the opportunity to write a rebuttal, it is helpful if you number your points of critique and your questions. They can then easily use the numbers to refer to and respond to these critiques and questions, saving valuable rebuttal space. Conversely, you can easily find their responses and check if they have addressed all your points.
It is also helpful if you indicate, at least to your fellow reviewers, but maybe also to the authors, what you took into account in your final assessment of the work. Especially if a paper ultimately is rejected, it is good to make explicit that this was, e.g., because the quality of the experiments was lacking and not because there were too many typos.
Be supportive
I firmly believe that if I help you thrive, it makes me thrive, too. I want my students to do well, because a) their success reflects well on me and on my university, and b) it’s simply much more fun if they succeed than if they fail.
Similarly, I want the papers at the conferences where I (try to) publish my work to be of high quality. I genuinely tried to come up with feedback that would make the next version of the work I was reviewing better, regardless of whether it would be accepted or not.
Naturally, it does not hurt to also compliment the authors on anything they did particularly well. We all struggle to get our work published, so encouragement and appreciation are always welcome. Furthermore, you will also benefit if people learn what makes a paper interesting or pleasant to read for you, so you can also compliment authors for thoroughly selfish reasons!
Be curious
Reviewing papers is a great opportunity to learn, which can be a great motivator to put some serious effort into it. You get a glimpse into the latest advances in your (or a related) field, which I trust you find just as exciting as I do.
On top of that, you can also learn a lot about other people’s writing process. While reviewing, ask yourself if the language and the structure in the paper works for you. Are there helpful analogies? Are the acronyms clearly explained? Is there a particular framing that you find compelling, or quite the opposite? Take notes and learn!
Be honest
Especially if it is your first time reviewing, you may not be familiar enough with all the material to assess its quality. Try to learn as you go, but also don’t overwork yourself. If you have reached the end of your knowledge, just be open about that to your fellow reviewers and area chair. Maybe they can fill you in, or at least take into account that your knowledge is lacking.
You may also not fully understand the process or the conventions, especially because not all of them are always clearly communicated. It is absolutely okay to indicate to the other reviewers and the area chair that you are unsure if a certain paper is out of scope or if certain conditions (like anonymity) are not met, as long as you indicate how and where you have looked for the answer and what the result was of that search. You may not be the only one who is confused, but you also do not want to cause a paper to be rejected just because something was not communicated clearly.
Be confident
Out of all the people they could have asked to review this paper, they chose you. You are part of the community. You have knowledge. You have ideas. You have a vision. You are qualified to give it your best shot. You have a lot to offer, so show it to them.
You are not the only reviewer and it is okay to disagree with the others. If, during the discussion, you find that somebody makes a good argument, let yourself be pursuaded and admit that you have changed your mind. Being able to change your mind when presented with compelling new information is a strength, not a weakness. If the new information is not compelling to you, it is also perfectly fine to not change your mind. You are entitled to your opinion.
Be a benevolent critic
This is really the summary of the points above. It is a phrase that prof. Holger Hoos (my PI) likes to use, and I find it very helpful. We also use it in our group, when we give each other feedback on our work. With any feedback I give, I ask myself if I am indeed being critical enough, and not slacking off or shying away from uncomfortable truths. Additionally, I ask if the feedback I give is constructive and respectful, and indeed likely to help the other person succeed, instead of demotivating them or dragging them down.
The award came as a complete surprise to me, if not to others. As one of my colleagues put it: “You cannot do anything without winning an award for it, can you?” Clearly that is untrue, but I am not going to deny that I wasn’t a bit flattered to hear that. Ultimately, it is nice to be appreciated by your community.
I feel like I just levelled-up as a scientist! Yesterday, the wonderful Daniël Fokkinga graduated with a very nice grade!
He contacted me in 2018, because he was interested in working on a project that I had proposed to help me integrate in the ADA research group, after they had ‘adopted’ me as a PhD student. The plan was to apply the paradigm of Programming by Optimisation (PbO) to an algorithm that my co-authors and I had developed for solving Stochastic Constraint Optimisation problems. Marie Anastacio would be my co-supervisor for this project, which was ultimately overseen by Holger Hoos and Siegfried Nijssen. Thus, I would have an opportunity to get acquainted with PbO, and with Marie and Holger, while at the same time helping a student graduate and do some cool, new science.
And that’s exactly what Daniël did! His work was accepted at the Data Science meets Optimisation workshop held in conjunction with IJCAI 2019 in Macau, and I had the pleasure of seeing him present his work there.
After a successful presentation and defence of his work in November 2019, he had completed all the work that he needed to get his MSc degree in Computer Science. Yesterday, we attended his graduation ceremony, after which we had drinks in the pub opposite of the Academic building in Leiden: Barrera.
I am very proud of my first Master student at @UniLeiden, Daniël Fokkinga (co-supervised by Anna Louise Latour, Marie Anastacio and Siegfried Nijssen), whose official graduation we celebrated today. 7.92 … very well done, Daniël – @ada_research will miss you! pic.twitter.com/qcGCCAJUgR
Yesterday was a very special day: Daniël Fokkinga, the master student who I have been supervising together with Marie Anastacio, Holger Hoos and Siegfried Nijssen, presented and defended his master thesis!
It is wild to see this happening only a few years after I defended mine. It feels like it was just such a short time ago that I was in his shoes, and yet I’m at the other side of this interaction now, prepping difficult questions for him to answer.
I was very happy to see that he brought a lot of friends and family, and it was lovely to meet them. Meeting Daniël’s parents and brother made me feel quite special, because he’s been such an important person in my work life for over a year, working with him on an interesting line of research. It was nice to have someone at the institute to talk to about my own research, and very gratifying to see him do so well at his presentation. It made me feel special that I got to share that with the actual important people in his life 🙂
Last week I attended the Benelux Conference on Artificial Intelligence and Machine Learning (BNAIC) for the first time. This year, it was held in Brussels, in my beloved Belgium.
This year’s edition was framed around AI Synergies: business meets academics. This was very well reflected in the excellent collection of demonstrations, where we learned about self-driving wheelchairs and puzzle-solving technology.
A wheelchair that automatically avoids crowds at #BNAIC. Interesting how they used standard PC mices to measure wheel movement pic.twitter.com/g2IZ8mdXqZ
My contribution to this conference was an extended abstract of this year’s IJCAI publication, in which we introduced a new propagation algorithm for stochastic constraints, implemented in the OscaR solver.
My presentation was scheduled in the very first session on the very first day. I was afraid that nobody would turn up, but luckily roughly forty people did. True, a lot of them were my amazing colleagues from the AIA group at UC Louvain, and a few of my (former) colleagues from LIACS in Leiden, but there were also a lot of very unfamiliar faces there.
Then, in my presentation, I used the example of a viral marketing campaign for chocolate, by means of ‘seeding’ certain people in a social network by giving them a free sample of the chocolate, and relying on them to promote the product to other people through word-of-mouth.
As a reward for the people who showed up at this early hour, I handed out Belgian chocolates to my audience. I hope they appreciated the joke 🙂
Since I’m spending this summer at UofT, I had to travel for just under 30 hours to get from Toronto to Macau, where I shared an airbnb with my colleague Marie and our master student Daniël. I arrived around midnight, in a 30-degree humid heat, with a 12-hour jet lag and still a bit flustered by the protests I encountered at Hong Kong airport. While the airbnb wasn’t particularly fancy (we are on a budget, after all) I was grateful for the AC, the cool shower, and the opportunity to check the wheels of my suitcase for crushed cockroach.
After a short night on my part, we walked to the west entrance of The Venetian: the hotel/casino that would host the conference. The Venetian was… an experience. During one of the breaks, I sneaked upstairs to check out the replica of Venice they built on the third floor. It was… an experience. All the stores and all the people made me dizzy, and the building seemed to have been designed by M.C. Escher: no matter where I went, I always ended up at the shops or the casino.
The canals and shops of Venice inside the Venetian in Macau.
I quickly learned that it was better to stay outside. The outside temperature never went below 30 °C, and with the high humidity, hot sun and occasional thunderstorm, most of us were soaked within a minute of leaving the venue. Therefore, we tended to stay inside, checking out the food courts of the adjacent hotels during our lunch breaks.
Palm trees and a fake Eiffel Tower
Upper level: shops. Lower level: casino.
Many locals travel by scooter.
All the kitsch and replicas, as well as the mimes in the Parisian and the singing gondoliers in the Venetian created such a surreal feeling of dreaming in me. A very weird place to go for doing your job as a computer scientist, indeed.
The Data Mining meets Optimisation workshop
The ADA research group sent a delegation of four researchers to IJCAI this year. Aside from Marie, Daniël and me, our professor Holger Hoos also came to Macau, just in time for the Data Mining meets Optimisation workshop at which our group presented no less than two papers!
After Daniël presented his paper (I am so proud!) and Marie presented hers (she has so much authority!) in the morning, Holger gave an invited talk about Cooperative competition: A new way of solving challenging problems in AI and beyond, giving us all food for thought for the reception afterwards. Of course, our mascotte, little Ada, was there to support us. Later, she came with us for a celebratory drink, showing her support for CLAIRE, too.
Little Ada Lovelace, our Lego mother of Computing, supporting the CLAIRE initiative. See how she resembles Leiden University’s Minerva?
My IJCAI highlights
One of my main highlights was getting to see professor Adnan Darwiche talk about his work. Throughout my short career as a PhD student, I kept returning to his work. In particular I have been building on his ideas of knowledge compilation: clever ways of representing solutions to problems that allow us to reason about those problems in a very fast manner. Darwiche inspires me as someone who, even at this stage of his career, still publishes papers all by himself. He is a great example to me, and it was incredible to get the opportunity to attend his talk on Reasoning about the behaviour of AI systems.
I was also very excited to attend talks by so many of the amazing fellow PhD students and postdocs I’ve met over the years: dr. Sebastijan Dumančić presented two papers in the Understanding Intelligence and Human-level AI in the New Machine Learning Era track and received an honorable mention for the EurAI Distinguished Dissertation Award 2018, and Alberto Camacho presented so many papers on Linear Temporal Logic (LTL) in both workshops and main conference, that I frankly lost count. Aside from these presentations, I also attended those of dr. Emir Demirović, Mohit Kumar, prof. dr. Lars Kotthoff and many, many more. All in all, there were so many presentations by these research friends, that they even overlapped, and I had to choose! (And no, I won’t tell you whose talks I did not get to attend 😉 )
This year was the second time that I attended the Women’s Lunch. I don’t know if this was intentional or not, but I got the impression that the organisers made a real effort to include the younger generation of female AI researchers in the community. Almost all of the senior women present at the lunch approached me to introduce themselves or to just have a chat.
Finally, of course, there was my own presentation. Unfortunately, I was awarded a time slot at the end of a session at the end of the day at the end of the conference, so the audience wasn’t as alert and attentive as I would’ve hoped. Despite this, they asked some interesting questions and laughed at my jokes, so that was good.
When I think back at how much effort went into this paper, it is hard to believe. I might actually write a blog post about it one day, for I reckon it could be interesting for someone outside academia to get a glimpse of what goes into publishing a bit of research at a conference. For now, the most important thing to say is that I feel very, very fortunate to have worked on this project with two of the most dedicated, smartest, kindest researchers I know: dr. Behrouz Babaki and prof. dr. Siegfried Nijssen.
Time for some R&R
Occasionally, we have a little time for being a tourist. I went to the northern part of Macau on two different evenings, sampling the local food. On another evening we went to visit the fake Eiffel Tower and looked out over all the lights in the city.
Little Ada making a new friend: Pollo.
Visiting the fake Eiffel Tower.
Taipa village at dusk.
Looking out over Old Macau (and in particular the Ruins of St. Paul’s) from the Monte do Forto.
On our last day we went to Macau tower for lunch. It has a rotating restaurant and we very much enjoyed the view, both of the city and of the people bungee-jumping off the tower!
View from Macau Tower.
After lunch we went to see the Guia Lighthouse and Lou Lim Loc garden. Unfortunately all in the pouring rain, but we still enjoyed the views and the turtles. We bought some bubble tea and took a taxi back to the hotel. From there, I took a bus to the ferry to Hong Kong, to continue my travels there.
Op vrijdag 16 november 2018 organiseren VHTO (Landelijk expertisebureau meisjes/vrouwen en bèta/techniek) en LIAXX (het vrouwennetwerk van de opleiding Informatica aan Universiteit Leiden) weer een Informatics Ladies’ Day (ILD)!
De jaarlijkse ILD laat meiden uit 5 en 6 VWO kennismaken met de studie Informatica en geeft ze inspiratie voor een carrière na het afronden van die studie.
Het programma is weer lekker gevarieerd dit jaar, met lezingen en workshops over onder andere: ethisch hacken, tekenen met Python, de efficiëntie en correctheid van algoritmes en hoe je robots leert lopen.
Daarnaast is er volop gelegenheid om kennis te maken met studenten en professionals in de informatica (zie ook dit voorproefje). Ik mag ook een halve dag meedoen! Komen, dus 🙂
