Recently, I had a chance to read and critique a paper as part of my school work. Reviewing that paper made me think about the philosophy of science. Why the authors thought their method is right and I think they made lots of mistakes in their methodology?!
The primary author has a Post-doc and the other one is a full professor, so how a grad student (whose mouth still smells milk - metaphor of being a baby in the academia) is giving such comments?!
In this post I'm not critiquing that paper, I'm just having a small discussion about some observations I had.
Before going more forward, I noticed that its more common in medical journals to publish a review paper that critiques other methods or projects and also report the observations from several published methods and papers together (secondary research).
If we look more close to the meaning of science, it has the idea of systematically understand different phenomena, organize them, find patterns and make predictions based on the what we learned and called scientific method. One of the take home messages here is that your method or experiment should be systematic and repeatable. My critique was the modeling method and procedure. There's a saying that goes like this: if you want to simulate it you need to be able to model it. Lots of our decisions today are based on the simulations which we constructed based on our models. Robustness and limitations of the model are the key factors for our results. One good example is what happened today. I got an email this morning at 4:30am from the Dean of our school which informed us that the school is closed today, however the snow started at 11am - we knew at least from yesterday with good confidence the probabilities of: how much is going to snow, when its going to start, when is going to end, wind speed, min and max of temperature, humidity (hourly)- good modeling btw).
One of the method I used for critique part of the paper had the idea of Meta-Analysis (which is more popular in statistics). Not really meta-analysis since I didn't have enough data, but I contradicted some of the facts in the paper by showing the disagreements from other sources (this doesn't mean that for example if I found 4 papers that approve case A and 2 papers that contradict case A, 4 and 2 are going to cancel out!!). However, we all know that the choice of technique, data availability, current status of concepts and case studies, statistical method options, strength of evidences, different types of errors ... can change the results.
My other comment had the idea of: Absence of evidence is not evidence of absence. In the aforementioned paper, authors concluded an event based on their hypothesis that their case (reforestation) will not work. I think they didn't look carefully and skipped the fact that what they actually can measure and decide. Sometimes we need extensive knowledge and keen power of observation.
My other critique was the information delivery or presentation of the paper. I was thinking about space limitation for presenting the finding and theory. I was also thinking about old movies, they were long. Directors had a good amount of time to include many details in their movie (like Lawrence of Arabia - almost 4 hours, think about it!). Current ones usually have more limitations (time-wise).
Papers in different journal have distinct limitations in the number of pages and words. The paper I was reviewing had a "Supporting Information" section in addition to the primary paper which is good for readers (and also writers) if they want to get more information and material. One of my comments was, why even though the authors had such a valuable addition and space, they didn't used it properly! If you don't have such a space, you should put your finding and theory more effective (this is different from effective theory). This means eliminating some of the data, body of knowledge and observations.
After all, I like to see papers, even though they have lots of mistakes or conclusions that contradicts our current facts. One of the reasons is the examples we had before. For instance, Newton's laws of motion (even though at some point we all know quantum physics are really in play) have no visible repercussions if for example we want to calculate our gas mileage using Newton's laws. Another example is what happened to Barbara McClintock. she stopped publishing her results because of derisions she received during here work. At the end, she won the Nobe Prize finally in 1983. But what it would be happen if the scientific community didn't judge her earlier!
- Positivism
- Forest Plot
The primary author has a Post-doc and the other one is a full professor, so how a grad student (whose mouth still smells milk - metaphor of being a baby in the academia) is giving such comments?!
In this post I'm not critiquing that paper, I'm just having a small discussion about some observations I had.
Before going more forward, I noticed that its more common in medical journals to publish a review paper that critiques other methods or projects and also report the observations from several published methods and papers together (secondary research).
If we look more close to the meaning of science, it has the idea of systematically understand different phenomena, organize them, find patterns and make predictions based on the what we learned and called scientific method. One of the take home messages here is that your method or experiment should be systematic and repeatable. My critique was the modeling method and procedure. There's a saying that goes like this: if you want to simulate it you need to be able to model it. Lots of our decisions today are based on the simulations which we constructed based on our models. Robustness and limitations of the model are the key factors for our results. One good example is what happened today. I got an email this morning at 4:30am from the Dean of our school which informed us that the school is closed today, however the snow started at 11am - we knew at least from yesterday with good confidence the probabilities of: how much is going to snow, when its going to start, when is going to end, wind speed, min and max of temperature, humidity (hourly)- good modeling btw).
My other comment had the idea of: Absence of evidence is not evidence of absence. In the aforementioned paper, authors concluded an event based on their hypothesis that their case (reforestation) will not work. I think they didn't look carefully and skipped the fact that what they actually can measure and decide. Sometimes we need extensive knowledge and keen power of observation.
My other critique was the information delivery or presentation of the paper. I was thinking about space limitation for presenting the finding and theory. I was also thinking about old movies, they were long. Directors had a good amount of time to include many details in their movie (like Lawrence of Arabia - almost 4 hours, think about it!). Current ones usually have more limitations (time-wise).
Papers in different journal have distinct limitations in the number of pages and words. The paper I was reviewing had a "Supporting Information" section in addition to the primary paper which is good for readers (and also writers) if they want to get more information and material. One of my comments was, why even though the authors had such a valuable addition and space, they didn't used it properly! If you don't have such a space, you should put your finding and theory more effective (this is different from effective theory). This means eliminating some of the data, body of knowledge and observations.
After all, I like to see papers, even though they have lots of mistakes or conclusions that contradicts our current facts. One of the reasons is the examples we had before. For instance, Newton's laws of motion (even though at some point we all know quantum physics are really in play) have no visible repercussions if for example we want to calculate our gas mileage using Newton's laws. Another example is what happened to Barbara McClintock. she stopped publishing her results because of derisions she received during here work. At the end, she won the Nobe Prize finally in 1983. But what it would be happen if the scientific community didn't judge her earlier!
- Positivism
- Forest Plot
No comments:
Post a Comment