Academic research: musings

[ariel]

Alan and Fernando,
Perhaps I did not myself clear enough.
I presented general “ academic “ rules of scientific research. And by “academic”
I did not mean that the research should be conducted within the ivory towers or by holders of PhD degrees.
Anybody could attempt to conduct research work. This can only be encouraged. But to be not only printable but also valid and respectable, the work should be conducted according to the general rules.

Thus, there are different levels of involvement in “research”. On the internet Fora, including the present one, most of communications are requests for help to demonstration of rare and potentially revealing examples and series of related examples demonstrating some similarities or time changes. The latter is as close to the “ hardcore” research as possible and should do its best to follow the “rules” ( sometimes it may be difficult to impossible, and the rest of participants should understand and accept it),

The next level is a “research article”. This is published in a journal or proceeding of a conference. Here adherence to the “ rules” is mandatory and external peer review is routine. Still, this may be a bit freewheeling” and can restrict its conclusions to “ offering a hypothesis” and opening it to critique and retesting.

The ultimate level is a full book. This is a summary of all we know for now about a particular field. It still can be offering hypotheses, but those can be presented in a non- biased and complete set of pros-and -cons. It requires as close adherence to the “rules” as possible, assumes critical analysis of the available data, use of statistics ( if possible) and of course absolute adherence to truthful information, without omissions of contradictory opinions and facts.
Description of examples should be complete, with references to other similar objects, analysis of inscriptions and decorations to tie them to other similar properly dated and attributed examples.

This is as concise as possible and we can expand our conditions . Can a book contain errors? Regretfully, no book will be free of them: the half-life of facts is a reality. But it should be as ideal as possible on the day it goes to print.

I know of several books from different authors and countries that qualify as “The Book” only by the number of pages and a hard cover. Some partially compensate by showing good images ( the so-called “ coffee-table” volumes) , many load as many identical examples without any analysis, believing that quantity is going to pad up quality. Some books are results of personal vanity of the author, intended of showing his/her collection before putting it for sale. By the same token, I know several catalogues of personal collections that unquestionably qualify as real academic works ( Moser, Buttin, Nordlunde, Hales, Pinchot’s catalogue of Wagner’s collection et cet.). Some are the worst : pure personal vanity ( “ I am not just a reader, I am a writer”) is the only driving force ( hint: jambiyas, but not by Gracie).

So, overall, there is a full spectrum of at least some contributions to real research: from asking a question the answer to which clarifies the issue and freewheeling exchange of opinions to full books addressing a wide swath of issues ( Rivkin and Isaac’s ” The study of the Eastern sword») as well as focussed on a particular variety ( Khorasani’s “Arms and Armor from Iran”»).

All are valuable to different degrees, but each next level requires stricter and stricter adherence to the appropriate set of the “Rules”.

Disclaimer: all names and titles are just the first ones to jump into mind. One can ask me about my particular opinion on other books, but I shall defer to the unified opinion of the Moderators.
No examples of European or Far( ther) Eastern topics were mentioned simply because I know nothing about them.

[A. G. Maisey]

Thank you for your further comments Ariel.

In my post #17 I asked this:-

"Ariel, if my understanding of that which you have written is incorrect I would appreciate your correction. It is important to understand how a message is to be understood before a relevant response can be written."

Since you have not addressed the matter of my understanding, I can only assume that you are in agreement with my understanding of your post #1.

In your post #19 you have expanded upon your ideas, but I do find that this new direction you have chosen does confuse me a little. I am now uncertain of exactly what type of research you are talking about, in your first post, you were quite specific in that your comments were focused on two separate, but related systems:-

a) academic research
b) scientific investigation

Now you have introduced the concept of "generality", and you mention "rules" several times. But then you move away from the two systems with which you opened discussion and you have diverged into another major system, the system of publishing the results of research. Academic Research need not be presented in any way, but the results of Scientific Research as a component part of the Scientific Method should be presented in one way or another.

I took your "musings" to be a fairly serious commentary on the research that often comes under discussion in this Forum, you wrote of the "rules of academic research", you asked if "our question is hypothesis driven", you queried the quality of the conclusions that had been reached. You drew parallels with your own professional work.

All of this is good, solid stuff. I can relate very easily to this, it is precisely the same sort of thinking that applies in my own profession, and exactly what governs the type work that I have done for most of my life. This work is audit, specifically operational audit, and using a systems based approach.

Before going into private consultancy I worked for a state instrumentality that was the owner/operator of a testing laboratory that was supposedly the largest and most highly regarded in the Southern Hemisphere, it might have been, it might not have been, I never researched the question. However, I was responsible for the audit of that testing laboratory, and from that involvement I also did private reviews of papers & articles produced by some of the people who worked there, written work produced by cadets who were working towards their first degree through to senior people who were engaged in post graduate studies.

So, coming from this background I read your initial post, and my impression was that you would have liked to see a slightly more disciplined approach to the research that is behind a lot of the discussion in this forum. My immediate thought was that yes, I can relate to that, let's not waste time on empty pipe dreams, let's focus a bit and produce some serious ideas.

But then I paused, and I considered the nature of this forum, and I concluded my post #17 by leaving the question open:- do we go the way of systematic research, or do we just wander around discussing generalities? Something to be said for both approaches, but I feel that in this forum perhaps a softer, more social approach might be the way to go. Polite discussion between gentlemen --- and ladies too, if we have any present, rather than incisive reasoning backed by dedicated systematic research.

Now Ariel, it seems to me that you moved somewhat from favouring the academic, scientific, hypothesis driven approach, and have moved towards a more "general" approach to research? Am I correct ?

In any case, we are still talking about "rules":- the "general academic rules of scientific research". But Ariel, you have not revealed those "rules " to us.

Please forgive me, I do not believe we are discussing the value or quality of any published works, publication of any research is a separate system to the research itself. The system of research might support the record of that research, but the publication of the record is a separate system. So, excluding all references to publishing of the results of research, and looking at only the concept of research itself, I believe that there are several terms that we need to clearly understand before this discussion can proceed.

The key words & ideas in your comments Ariel, appear to be:-

a) research, this word is from the word "search" to look for something, we all carry out research constantly, we want to buy a new car, so we research brands, models, prices; we want to buy groceries, so we research various stores for range and prices. This is general research, just looking for the best deal, or the most suitable solution.

b) academic, when we couple "research" with "academic" we become a little more specific, by definition, "Academic Research" is no longer just "research" it is a particular kind of research, and it is governed by rules, in broad terms the applicable rules are that Academic Research must employ a systematic approach and have an objective.

c) scientific, when we couple "research" with "scientific" we become even more specific, "Scientific Research" is a sub-system of the "Scientific Method". I have encountered a number of definitions of the component sub-systems of the system known as "the Scientific Method", the progression of this method of investigation looks something like this:-

question> research> hypothesis> test> observation> analysis> conclusion> presentation of findings

now, strangely enough this is a precise parallel with the methodology of Systems Based Audit, something that I have now been using for around forty years.
So, Scientific Research shares its nature with Academic Research, both are based upon a systematic approach, both have an objective, but Scientific Research is a part of the Scientific Method, which has as its ultimate objective the presentation of findings from that research, the findings from Academic Research might never be presented in any form.

d) rules, that is, the rules which govern both Academic Research and Scientific Research, well, the actual rules are the same:- the research must be carried out in a systematic way and that research must have an objective. It is a given that according to the field in which the research is being done, the elements of the system used and of the objective will vary.

The take-away from this is that once we introduce the concepts of "academic" and "scientific" the idea of "general" can no longer apply:- general research is what we all do constantly, Academic Research and Scientific Research both a have the requisites of system and objective.

We can research things in more or less general way, or we can research things in a structured, systematic way.

So Ariel, my question now is this:- as a standard for this forum, do you favour the more or less general form of research that is a pretty comfortable foundation for social discussion, or do you favour the hard edged systematic approach that can generate learned discussion and possible conflict?

Personally, I'd rather go with the soft approach.

[ariel]

Alan,
Sorry if i did not respond t your earlier quesion. I kind of missed it within you message.
Your current one is clearly well demarcated and stated. So, here is my my response:

Our posts on this Forum are by and large a mix of information search and social interactions . As such, they are practically immune to any critique, and definitely to any harsh one. Reasoned disagreements are part and parcel of any discussion of any object or opinion ( “You are mistaken, in country A it is called not B but C. And here is the reference”), but sarcastic remarks, personal insults and such are unacceptable and the moderators must be harsh with the offender.


This is my view of a soft approach: friendly, respectable, sincere ( without being denigrating) and forgiving. In short, kind of family chat at the New Year table ot just Sunday dinner. By and large we practice it already, so it is not a great burden to continue.

Things change if one of the Forumites posts an article -like treatise on the Forum, with the reason for doing it, data, discussion with references. This is a different ball of wax and should be treated as a scientific ( or “academic”, if you prefer) contribution. The contributor decided to produce a serious piece of work and the greatest compliment from other Forumites is to take it seriously. Sometimes, the post contributed to the Forum is an early draft of a journal paper the contributor reveals to his friends (sic!). Their frank comments may be immensely helpful. I know it very well: I often send my own “ main job”:-) papers to friends for their perusal and very (!) frank comments. Saves me a lot of grief with the subsequent journal referees. My ego may be bruised, but my paper improves markedly. I also get similar requests from friends and do my best to return the favor.

This is a harsher, but still friendly approach.

The last one deals with articles and/or books submitted for publication elsewhere and peer review or published without the “ pre-screening” whether by Forumites or external colleagues. Here , if we want to be truthful and the topic is important to us we might go full forward and provide as harsh critique as the contribution deserves ( in our opinion, at least).
Again, no insults are allowed, but sarcasm is permitted when it is appropriate.
This is the “ harsh approach”. It should be factual, but pitiless.

This is the usual policy of all professional journals I know and definitely of the two I am editing now or edited in the past.

This is to prevent false or bad information to pollute the pool or to alert colleagues that a particular piece of information had just hit the road and is poisonous.

The entire medical field is functioning now on the principle of “ evidence-based medicine” . This is the beginning of the end of “ personal opinions”, poorly-executed studies, falsified results etc. I personally know people whose careers went down the drain because they published 2 pictures of the same histological slide turned 90 degrees away from each other and labeled as showing different phenomena.

Every submitted paper undergoes computer analysis to uncover plagiarism.

Every case of “ double publication” results in published retractions, indefinite ban by both involved journals as well as by a slew of others at the very least by the same publishers.

I see nothing wrong in implementing the same harsh policies in other fields.

Hope it answers your question. There is no “ all or nothing” cookbook, it all depends on the circumstances.

[A. G. Maisey]

Thank you for your response Ariel.

I believe that you & I are on the same page in respect of the standards for discussion on this forum. I can find nothing with which to disagree in your response. Very nicely stated.

However, I do feel that by moving into the area of publication you have moved away from the core function of this Forum. I do agree that prior to the publication of just about any serious work, and most especially serious work that is also original work, that work must undergo review at more than one level.

The reviewers should not mince words and should not hold back on anything. But I do not see this Forum as the place for such review. I believe that the people to carry out pre-publication review must be carefully chosen by the writer. Those people should be chosen upon the basis of their relevant knowledge & experience in one or more aspects that apply to the work to be published.

I do not see an online forum, especially a forum that is open for perusal by the entire world, as a place for such review.

You began this thread with some comments on "research", I think it was I who then diverged into one of the products of research, and addressed the desired or preferred nature of discussion, and you have answered my question well.

However, we are still left with a comment from your opening post, and for me, the questions raised by this comment are at the heart of this current discussion:-

"--- What all of us need to remember that there are rules of academic research, irrespective of the topic. Is our question hypothesis driven? How solid is the hypothesis? How do we plan to prove it? What kind of analysis are we going to employ? How stringent are we going to be with our conclusions? Will our conclusions add something important to the existing body of evidence? "

Can we understand your remarks here to apply to only the more serious contributions to our forum discussions, or should we use a structured, systematic approach to all of the seeking after information that we might undertake?

I feel that it might not be a real bad idea to leave the terms "academic" and "scientific" behind. Even amongst the academics & scientists who populate the communities of academia & science it would seem that there is not always a uniform understanding of the meaning of these terms, and amongst lay people, I would suggest that there is even less understanding.

So my question now is this:-

in your opinion, in this Forum, is it acceptable for random, unstructured ideas, observations & comments to form a basis for discussion, or should we try to always use a systematic, structured approach to enquiry, prior to presenting our ideas, observations & comments?

Can we understand that your comment that I have quoted above is to be applied to all data included in our posts, or should these concepts of system & structure only apply to those instances where a contributor to this Forum has posted information based upon what he or she considers to be serious research?

[Jim McDougall]

All this from a simple query asking for images of an Indian warrior wearing a khanjharli dagger, and an accompanying photo with a Sikh said to be with one.

Personally, while i do not have formal education training in procedures and methodology in systemic research...I just enjoy study, and learning from my own style of research. With this it is quite comfortable and rewarding and I enjoy the discussions being the same.........though my writing is typically a bit heavy and detailed. Most of that is simply because it is my way of compiling often notable time in research.

I dont think a private venue/forum needs to follow the strict regimens of academia to be useful and advance our collective knowledge on weapons in forms and history. We all have different perspectives in our interests in arms & armor.

While I admire the disciplines and achievements involved in academic pursuits and the structured scholarship, and all of those here who have accomplished notable credentials and experience.......I also enjoy just 'talking weapon study and history'.

In answering the question in the original thread, there were no protocols or structured methods followed...simple 'cerca trova' .
In the interpretation of this as I have understood....'seek and ye shall find'.

[Edster]

Thanks Jim, excellent counterpoint. I, too, enjoy the entertainment quality of good reasoned debate. However, as an example, I would also like to know the objective differences between a German made 19th C. munitions grade trade blade and those made in Kassala in the early & mid 20th C. Without makers marks most look the same to me.

Best regards,
Ed

[Jim McDougall]

Originally Posted by Edster

Thanks Jim, excellent counterpoint. I, too, enjoy the entertainment quality of good reasoned debate. However, as an example, I would also like to know the objective differences between a German made 19th C. munitions grade trade blade and those made in Kassala in the early & mid 20th C. Without makers marks most look the same to me.

Best regards,
Ed

Thanks Ed,
Actually when I began writing on these forums over 20 years ago, Mostly I just wanted to learn from other guys involved in studying the same weapons I was interested in. I have never cared especially for debate, but in sharing information and observations with examples and evidence.

I always appreciate when an example or observation is shared and when contrary views are shown, with explanations and details. Rather than debate these are discussions evaluating information at hand and constructively compiling material to advance the collective knowledge of all reading.

Great example on those Sudanese blades, I am always baffled at trying to evaluate them without markings as well. We often presume a blade must be European , but these guys in Kassala were pretty good.

[Edster]

I'm a perennial lurker on this Forum and read most all the posts. Yogi Berra once said "You can observe a lot just by watching". This is what I have observed.

In my view we're are mostly asking Cultural Heritage Research questions related to metal objects (weapons): What is it? How, Where, When and sometimes Why was it made. As previously stated we usually rely of soft methods based on personal observations and experience to offer reasoned opinions.

How do we really tell the difference between historical native blades and European or Asian imports? Where did native iron or copper actually come from? Composition of twist core, laminated or wootz blades? Hardness results from native vs factory heat treating? (Are testing files good enough?) Is that coin silver or from sterling? (Chemical test is available for silver.)

Unfortunately, we lack hard data developed via scientific investigation. That's not really our fault because we don't have access to expensive scientific equipment usually only available to industrial facilities, museums and research universities. Non-destructive tests are available of only we had access to them. Fortunately, portable X-Ray Fluorescence testers and even Neutron Imaging techniques are becoming available to test for metal alloy compositions.

How do we gain access to these equipment so a database can be developed? I tried to get a material science prof. to help analyse my Kaskara blades. He didn't respond. Likely, I didn't sell the idea properly. Maybe we can develop cultural heritage research projects and pitch them to nearby universities or museums; make it interesting and publishable to them. Maybe do a GoFundMe account to do real analysis?

Anyway, I like to see we forum members work toward more reproducible data to inform our collective judgements.

Best regards,
Ed