DailyDirt: Judging The Quality of Science
from the urls-we-dig-up dept
The Scientific Method is often taught as a linear process that proceeds from hypothesis to theory. In practice, science -- like any other human endeavor -- can be much more haphazard. Science isn't perfect, but it has some built in processes for error correction to help weed out mistakes and outright fraud. There are some well-known issues with the traditional peer review mechanism, but science is adapting and coming up with new ways to improve its results and conclusions. Here are just a few links on judging the quality of science.- The Meta-Research Innovation Center at Stanford is planning to develop a meta-research institution to research... research. The new lab will be called METRICS for short, and it'll study factors like irreproducibility and publication bias. [url]
- Jimmy Wales has decided that no pseudo-science will be accepted for publication on Wikipedia. Wales says, "If you can get your work published in respectable scientific journals -- that is to say, if you can produce evidence through replicable scientific experiments, then Wikipedia will cover it appropriately." [url]
- As more scientific research is done more globally, is the diversity of the researchers influencing the quality of the results? Some analysis suggests that papers with geographically diverse authors are better, in the metric of getting more citations. [url]
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: jimmy wales, meta-research, peer review, publication bias, reproducibility, science, wikipedia
Reader Comments
Subscribe: RSS
View by: Time | Thread
Science or Pseudo-Science - How can we tell
The biggest problem we have on both sides of the argument is, of course, people. Now that I have more time to look into various subjects, it is interesting to see the number of main stream areas (in science) that are actually problematic. Particularly, when a small group sees one of the problems and starts to discuss this and are then labelled as "kooks" because they do not agree with the mainstream ideas. Yet they are raising legitimate concerns about the prevailing models.
Of course there are many different models out there which fail even the first experimental tests. But there are still others which oppose the standard models and have suitable match up with experimental results and still are considered pseudo-science because they go against the mainstream.
I know of various experiments that have been proposed which have been given no support at all. These experiments would categorically dispose of the alternative model if they failed. Yet as far as I can tell, no such experiments are being worked on because it would be career suicide to partake of these experiments. It quite often appears that those controlling the purse strings are the ones who would lose most with their reputations if they allow the experiments to go through.
[ link to this | view in thread ]
That should be easy...
You mean like this?
[ link to this | view in thread ]
Re: Science or Pseudo-Science - How can we tell
Yes, science is fallible, because it was created by fallible humans, after all. But, rather than relying on some imaginary Higher Authority to fix the problem, science, builds on an interesting characteristic of that fallibility: that we are better at spotting other people’s mistakes than our own.
This is the essence of critical thinking. Not only is it useful to help filter out mistakes, it also works against deliberate fraudsters as well.
Perhaps that’s why the “alternative medicine” crowd—and all the other religionists—hate it so.
[ link to this | view in thread ]
Re: because it would be career suicide to partake of these experiments
We saw this with the slow (decades-long) mainstream adoption of quantum physics, for example. I forget which physicist it was who said “science progresses, one funeral at a time”...
As for funding, look at all the money being freely bestowed on out-and-out crackpot ideas. Surely something with even the remotest sliver of a chance of success would be just as likely to get some rich backer.
[ link to this | view in thread ]
Re: Re: because it would be career suicide to partake of these experiments
I have seen enough discussions that would strongly suggest otherwise. The generally espoused comment is that you have to toe the line of what is acceptable or you won't get funding/tenure/etc. Cross the line of looking at something the bossman doesn't like and you are in for it. This happens in all areas wether it is business, research, design and development or any other area.
Look into the political affiliations and you will see the reasoning for the backing. What I am talking about is funding from within the halls of science (universities/research centres/etc). Areas that will cause a problem for the purse strings or go against its belief system will not get backing. Again I have seen this is various areas in business, universities, etc.
[ link to this | view in thread ]
Re: Re: Science or Pseudo-Science - How can we tell
Let's apply this to astrophysics. Black holes, neutron stars, dark matter and dark energy. None of these entities have actually been observed - they have been proposed based on the mathematical models developed and are believed to exist. But they have not actually been observed as testable entities.
Yes I know there will be those of you who will say that they have been observed. But I would suggest you check out the actual language used and it boils down to "most probable object/entity". If you pin these scientists to the wall they will say that they believe the entity in question is one of the kinds of objects. Dark energy and dark matter have most definitely NOT been observed. Every experiment that has been devised to find them has had a negative result.
True, they rely on their Higher Authority to determine the "truth".
There are interesting discussions going on at the moment about the difference between "science" and "scientism". Whatever your view, there are some very critical thinkers who are raising their voices about the rise of "scientism" and the harm it is causing to the scientific method and investigation of the world around us.
I have had my discussions with various "subject matter" experts over many years about different models. There have been various attitudes expressed when asking questions that oppose the "models" believed in by these experts. Everything from "good questions and here is my take on the answers" to "trust me, it's too complicated for you" to "you're an idiot to be asking any question against the model - it's proven fact".
Which religionists? - atheists, buddhists, moslems, hindi, communists, fascists, humanists, naturists, christians, capitalists, new agers, alienists, nihilists, pantheists, animists, secularists, etc.
[ link to this | view in thread ]
Re: What I am talking about is funding from within the halls of science
[ link to this | view in thread ]
Re: None of these entities have actually been observed
[ link to this | view in thread ]
Re: You mean like this?
[ link to this | view in thread ]
Re: Re: Re: Science or Pseudo-Science - How can we tell
I don't know what you mean by "observed as testable entities". Our theories of these entities make predictions, which we can and do test. If you don't want to believe in them, fine, don't. But I have a sneaking suspicion that the "models which go against the mainstream" -- which you mention but don't specify -- don't stack up so well.
[ link to this | view in thread ]
Re: Science or Pseudo-Science - How can we tell
Umm... All of that would apply to an experiment like "let's launch an elephant into orbit to test the theory that elephants can survive unprotected in outer space".
Please, tell us about one of these proposed experiments. Be specific.
[ link to this | view in thread ]
Re: Science or Pseudo-Science - How can we tell
This is not a new development, it is possibly human nature.
For example, in the past mainstream thinking put the earth at the center and everything revolved around it. This has been proven to be incorrect and yet there remains a few who espouse its veracity.
Do you have any examples of these kooks and their pseudo-science? Generalizations and vague declarations do little to support a contention.
[ link to this | view in thread ]
Re: Re: None of these entities have actually been observed
[ link to this | view in thread ]
Re: Re: Science or Pseudo-Science - How can we tell
Nobody else brought up religion, so why did you?
I mean, can we not discuss science without the "at least it's better than religion" argument popping up? Seems like it happens every single time. And most of the time it's in the form of exactly this sort of sneering, throwaway remark.
It's just tiring.
[ link to this | view in thread ]
Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
[ link to this | view in thread ]
Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
Black holes - include time dilation effects and they will never form
Neutron Stars - include strong force interactions, a neutron star would have to be considered as a single nuclear entity and by the rest of nuclear physics cannot exist. Gravity is not sufficiently strong enough to override strong, weak and electromagnetic forces at the atomic and sub-atomic level. Remember that neutrons themselves are unstable particles unless found in very specific environments (which require protons and electrons).
They only exist if you ignore the full range of effects and forces as predicated by the various models in use.
I don't deny that there are strange entities out there but I do seriously doubt that they are what the current ideas are saying.
[ link to this | view in thread ]
Re: Any science classroom microscope is powerful enough to see them.
[ link to this | view in thread ]
Re: when you include all model phenomenon for these entities, they can't exist in our universe.
How do you reconcile that? Perhaps your (mis)understanding of physics is the problem.
[ link to this | view in thread ]
Re: Read the article carefully,
[ link to this | view in thread ]
Re: Nobody else brought up religion
[ link to this | view in thread ]
Re: Re: What I am talking about is funding from within the halls of science
One of the more devastating attacks against any theory is to say it is not even wrong. So tell me how science deals with the truth? The problem here is that testability requires finding ways to prove it wrong not prove it right (not prove it true).
Unfortunately today, there is a mindset that makes the assumption that science can prove things true. This mindset turns science and the scientific method into a religion instead of it remaining a tool for investigation of the universe around us.
[ link to this | view in thread ]
Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
Both the climate-change denialists and the anti-evolutionists despise scientific modelling.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: when you include all model phenomenon for these entities, they can't exist in our universe.
It is a simplification (an approximation) of the actual entity or physical environment. Many things are ignored to obtain the model. The model is used for specific investigations. So when you change what parts of the actual entity or environment you are investigating, then a different model will generally be needed (so you can ignore areas you are not investigating).
I have also seen a review of said work that says the model being by him used was based on a single body within the universe, not multi-bodies that exist in our universe.
Again, I'll reiterate - models are simplifications and approximations of the real world. Go outside the boundaries of the model and the models will give you incorrect results (incorrect approximations to the actual answer).
Example: Newton's Laws of Motion (including gravity) are very good approximations (a good model) at low speeds and low gravity but at high speeds and/or high gravity corrections have to be made to the model to give the answers which approximate the actual observed results.
Example: Electronic circuit analysis at DC or low frequency AC uses simplifications and approximations of the different types of circuit elements. Change the conditions to high frequency, the model must change to give you the results.
Example: Static and Dynamic analysis of structures. Assumes in many cases that we are dealing with rigid bodies and for the level of forces and the frequencies in question, we can do this. But change things and we need to change the models to non-rigid bodies.
Example: Mathematics uses infinites and infinitesimals to provide a way of solving calculus problems. There are different techniques to remove infinites from the results including the old L'Hopital's Rule. The mathematics is then used in the various models for the discreet world.
I understand what models are and how they are used and I don't assume that the model in question is relevant outside its original domain.
[ link to this | view in thread ]
Re: Re: Read the article carefully,
Now if such terms are used in other fields, then it becomes acceptable to have a dissenting view.
After 20 to 30 years of reading and trying to follow the mathematics involved in various models that deal with both sub-atomic and galactic/universe, I want to know what are the basic assumptions that they start their models with.
An interesting phenomenon that occurs in any model development is that certain things are ignored when first developing the model, which may be included later.
If these elements are included at the beginning, the end model can be quite different and the outcomes expected from the model quite at odds with the original development.
[ link to this | view in thread ]
Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
An additional part of the initial model is that there is also terms which may relate to momentum effects. But further investigation is required and experiments need to be fully devised and done to see if the model has any opportunity of merit.
[ link to this | view in thread ]
Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Religion and Philosophy deal with the Truth.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
[ link to this | view in thread ]
Re: Re: Religion and Philosophy deal with the Truth.
Yes there is only one reality. However, as you are very finite and very small (like everyone else on the planet), you are unable to comprehend reality in any very meaningful way, let alone in its fullest extent. That is why we have unanswerable questions. That is why there are different views about the universe and the meaning of life.
The amount of knowledge and understanding we now have is insignificant to the amount of knowledge and understanding that is available.
My own perspective of what I see of the universe leads me to the conclusion that it was created. That perspective also leads me to want to understand (as far as I am able to) the various aspects of the universe. That perspective also leads me to the understanding that I will never be capable of understanding it, but that there is a great deal of enjoyment in trying. That perspective also leads me to want to know who created it and why.
Science and the scientific method are just one of the tools available to me to gain understanding.
YMMV.
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
Have you looked into the model in question? Have you attempted to use Mathematica or Maxima (or other products of their ilk) to do an analysis of the model? Have you thought about what experiments could be undertaken that might shed light on the validity of the model?
If not, why not? If so, what are you results?
If you haven't actually looked at the model in question, then you have not been able to address its short comings or assumptions. If you have looked at the model, what are the shot comings and assumptions that you have found?
[ link to this | view in thread ]
Re: Re: Re: Re: Re: Science or Pseudo-Science - How can we tell
What, exactly, are you attempting to accomplish in your many comments to this thread? It appears you have some grievance with the treatment of one particular theory and then claim all science is therefore suspect as a result. How very unscientific of you. Do you have other examples where your pet theory has been trashed by scientific gang members?
[ link to this | view in thread ]
Re:Show me any experimental evidence that a multi-neutron entity (where the number of protons is very much less than the number of neutrons) exists and is stable
[ link to this | view in thread ]
Re: you are unable to comprehend reality in any very meaningful way
[ link to this | view in thread ]
Re: Have you seen the proposed mathematical model
[ link to this | view in thread ]
Re: Re:Show me any experimental evidence that a multi-neutron entity (where the number of protons is very much less than the number of neutrons) exists and is stable
I want to see a stable entity that has a ratio of at least 100 neutrons to one proton.
The last stable element is Pb (Lead), though this is still considered very slightly radioactive. The neutron to proton ratio is less than 2:1.
Neutron stars are considered to considered to consist of neutrons with a very small level of impurity by protons, hence I want to see a stable entity that has at least 100 neutrons for every proton.
[ link to this | view in thread ]
Re: Re: you are unable to comprehend reality in any very meaningful way
[ link to this | view in thread ]
Re: Re: Have you seen the proposed mathematical model
The model I have referred to looks at the dynamic consideration of opposite charges oscillating in their relative positions, which gives rise to different results. It is the residual effects that are being considered and they are inverse distance squared and very, very tiny on the order of the ratio of gravity to electric field strength and always attractive.
Again, to get back to the point being made. This is an alternative model for the effects of gravity for which various experiments can be formulated that would either show the model is defective and hence not worth looking at or produces results in accordance with the model that would allow further study to be undertaken and hence further experiments to be developed.
This is the application of the scientific method. To simply throw the idea and model out without at least doing some initial experiments that are proposed by the model because it disagrees with the currently accepted theories shows a fairly rigid and closed mindset.
[ link to this | view in thread ]
Re: Neutron stars are considered to considered to consist of neutrons
[ link to this | view in thread ]
Re: Let me ask you a question in turn
[ link to this | view in thread ]
Re: Re: Neutron stars are considered to considered to consist of neutrons
Since combinations of neutrons in a neutron stars would not be in stable nuclei, we should expect to see significant decay, hence proton/electron formation. What is the effect of the strong/weak/electromagnetic force on the stability of the entity in question? What happens to any electric currents that may form in any magnetic fields that may be created?
The model posits that the strong, weak and electromagnetic forces are non-existent and that the only consideration is gravity. This is not demonstrated as a factual reality. We see all sorts of interesting phenomena in ordinary radioactive materials, so why would we not see such in the posited neutron star?
My whole point is that treating an entity that is effectively only neutrons (as per the model) and as such is not "ordinary" and is only affected by gravity (as per the model) is so simplistic as to be effectively nonsensical. Those who support this model would have to demonstrate a number of things:-
1). Entities consisting of either only neutrons or having "impurity" protons of less than 1% exist.
2). That such entities are only affected by gravity alone and that the strong, weak and electromagnetic forces are irrelevant to such entities.
Now it may be that someone somewhere has done both things as well as any other necessary provisos for neutron stars to exist as the model states. If this has been done, I would have expected it to have been heralded from the rooftops. Have you heard of such news?
[ link to this | view in thread ]
Re: Re: Let me ask you a question in turn
Show you perspective, otherwise, as has been demonstrated by your various responses, you are unable to critically answer or present an opposing view. You have only thrown "potshots" so to speak. I have taken time to answer you questions, but up to this point, you have not been able to show where my reasoning in incorrect. Nor have you presented anything that says you have considered the problems inherent in the models and have solutions that make the models still valid.
If you have such, present it, otherwise I will take it that you know that you have nothing and are simply using your conversational method to hide that fact.
The ball is in your court. Enjoy your day, and I look forward to reading your discourse on the subject.
[ link to this | view in thread ]
Re: The point I am making is that gravity is completely insignificant at the level of neutrons beside neutrons
[ link to this | view in thread ]
Re: The onus is now on you to present an alternative
The answer is, you’re not. Your arguments fall at the first hurdle of critical thinking.
[ link to this | view in thread ]
Re: Re: The point I am making is that gravity is completely insignificant at the level of neutrons beside neutrons
You appear to have an inability to follow a discussion in any logical form but appear only capable of approaching the discussion from a point of view that is based in total belief that you are right and anyone that has an alternative view is wrong. This is demonstrated by
You do not even question if neutronium is even possible.
This is usually the argument that is made against religionists of any kind. So, the question is - are you a "true believer"?
[ link to this | view in thread ]
Re: I am not saying neutronium exists
[ link to this | view in thread ]