The Eblan Thaw Disaster

There are strict processes I follow when testing anything for fishing.

There are substantial changes made depending upon exactly what is being tested, but constants and factors when doing Cast quantitity tests are:

Everything done is recorded in order to be re-watched for verification, or streamed if there is not room on my HDD.
A private linkshell or FC chat is used to keep track of things and for documentation of when important things happen with video timestamp to easily refer to when the recording is done.
All abilities that have the potential to interfere with accurate data collection if used accidentally are removed from the hotbar while testing.
All fish are kept in inventory or FC chest tab 5 until testing is complete for easy verification of quantities.
Data is tracked in a spreadsheet in real-time, and then crosschecked.

In the middle of june 2021 I was working on the Big Fish title on an alt that had not done the Facet quests and got curious how much of an advantage it was not having Amber Monkfish available in The Hills of Amber. I did a small amount of testing and the statistically insignificant data I collected didn’t look interesting enough for me to care about at the time, so I didn’t continue.

In Endwalker, I was curious again about the fish added to the holes when doing the Studium quests. Because Garlemald only has two fishing holes, and the quest fish was available in both, this meant the Studium fish was certain to be in the hole with Garlemald’s legendary fish in the future.
In december 2021, I began fishing in The Eblan Thaw with Mayfly idly as I felt like it and kept track of data in a spreadsheet as I normally do. I chose The Eblan Thaw and not The Stainless Course because it is closer to the aetheryte.
At this time I was also doing significant testing on the scrip fish. Scrip testing was much more important to me, very time intensive and also inventory intensive, so I sold my fish from The Eblan Thaw each time I was done fishing for a session.
In total, I collected data for over three weeks in this haphazard manner while I was working without recording the process as I already had significant purple scrip video to review. This is not unusual when it comes to things I’m curious but not serious about, to do lazier work to get a preliminary look and then to return when I’m able to dedicate serious effort if I feel like it. There is a significant amount of things I’ve looked into and left unfinished or poorly done and not shared the data for as it was done in this fashion. I keep my well-verified and less/un-verified data in separate places.
After I was done the data from The Eblan Thaw stayed with my less/un-verified data and I didn’t think about this testing again until this patch, when we got the legendary fish.

2021 data:

2024 data:

The 2024 data is easily verified. I recorded the entire process and completed it over the course of 8 days (30/01/2024 – 07/02/2024) while fishing nowhere else on the character without Studium progress and never touching the inventory except adding Mayflies. On the character with Studium complete I fished in in Sidereal Whale, Furcacauda, or Hyphalosaurus windows and did not touch the inventory other than to sell fish from those windows.

The characters upon completion of their respective 5000 catches:

The 2024 results, and similar quantity testing done by a twitter user, indicate the 2021 data is wrong.

My question is: where?
The answer is: It ultimately does not matter and I am unable to answer with any certainty because of the lack of QC I employed.

Knowing myself and being familiar with my weaknesses and typical mistakes, it’s very likely just a data entry error, or multiple of them.

While entering data periodically before being done fishing for a session, I often catch myself entering in numbers incorrectly, though I also usually notice right away – but not always. This is usually never a serious issue with something I will be sharing publicly. With anything else I have shared publicly, I can either verify via recording the precise quantities at any given time, or the totals via the amount of fish in my inventory.
I’m certain I went to correct any errors I made mid-session before selling my fish in 2021, but even when working with correct numbers it is possible to manually enter something that is incorrect. Errors like the one below were made during the 2024 test.

I also also on two occasions entered data for Before Quest into the same fishes field for After Quest, or the opposite.
As stated, these are mistakes it is entirely reasonable to make and ones which I have a tendency to do. Reality is, much of large-scale individual FSH testing is very boring and tedious and lends itself to such errors.

It is easy to keep accurate counts when you maintain two means of keeping guaranteed accurate count.

This is why I take the measures that I do, and why I force myself to be very careful with what I share.

That I shared the 2021 data was a much larger error on my part than whatever error(s) I made in collecting the data, so the exact nature of the error(s) in the 2021 data is irrelevant.

Conclusion: Mistakes are not acceptable, but understandable.

Though this is a hobby, it is one I take seriously. Data or research I provide is that which is done by an individual and is not peer reviewed. Due to this, I take what I consider to be strong measures to ensure that what I provide is accurate, and this is where the issue lies.

I was aware that I could not validate the data I had. That should have prevented me from sharing it.
There have been several instances where I did not provide information or data I had collected because of similar issues.
Making a mistake in the research or data collection process is fine. When I become aware, or suspect, that I made a mistake, I determine where the error came from and correct it if possible. If this is not possible, the existing data is disregarded and the test is redone. This is also possible if I don’t think I made a mistake but it is possible that I could have. This is time consuming, but it is the responsible thing to do.

I was given a small amount of statistically insignificant data from other parties with numbers that were similar to my 2021 data. If one is a poor researcher, this could have been seen as validating my data.
The 2021 data had also indicated certain numerical scaling, so when viewed while already seeking a specific conclusion “made sense.”
Neither of these two things are sufficient replacements for the QC which is normally employed, and is extremely poor research practice.

My personal life has been intensely hectic and my free time/access to a computer limited, so I was not going to have time to redo the test immediately. This is understandable.
I shared the information against my better judgement with the intention to redo the test as soon as I was able. This is unacceptable.