Open source

Message boards : Science : Open source
Message board moderation

To post messages, you must log in.

AuthorMessage
Webysther

Send message
Joined: 14 Oct 14
Posts: 3
Credit: 91,655,456
RAC: 0
Message 2523 - Posted: 20 Apr 2020, 4:27:02 UTC

Is there a chance to make the source code available in some way, as in SVN, git (Github) or zip? Thank you.
ID: 2523 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Vester

Send message
Joined: 19 Apr 20
Posts: 11
Credit: 252,179,793
RAC: 0
Message 2525 - Posted: 20 Apr 2020, 11:24:13 UTC - in response to Message 2523.  

ID: 2525 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Webysther

Send message
Joined: 14 Oct 14
Posts: 3
Credit: 91,655,456
RAC: 0
Message 2533 - Posted: 20 Apr 2020, 23:56:36 UTC - in response to Message 2525.  

Thanks Vester, but I'm talk about the source code of this project.
ID: 2533 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Vester

Send message
Joined: 19 Apr 20
Posts: 11
Credit: 252,179,793
RAC: 0
Message 2534 - Posted: 21 Apr 2020, 0:47:48 UTC

This link isn't what you need either but it is interesting.
ID: 2534 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Webysther

Send message
Joined: 14 Oct 14
Posts: 3
Credit: 91,655,456
RAC: 0
Message 2562 - Posted: 26 Apr 2020, 22:07:47 UTC - in response to Message 2534.  

This link isn't what you need either but it is interesting.


I still wait for admin to know if has any intention to make the source code public, for me is important because in recent future I will create a FPGA using AWS structure to create a optimized version for some tests, the cuda code could help me to reduce the time to create this PoC.
ID: 2562 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
poppinfresh99

Send message
Joined: 25 Jan 20
Posts: 57
Credit: 1,225,329
RAC: 0
Message 2600 - Posted: 1 May 2020, 23:58:12 UTC - in response to Message 2562.  

Yeah, for sure, put the code (at least all the code needed to run on OpenCL GPUs) on GitHub so I can take a crack at making it many times faster! Just do it soon so I can look at it over the summer.

Instead of people optimizing their hardware to squeak out a couple more calculations, let's see if we can't improve the algorithm to make everybody's hardware run MANY times faster! This type of optimization would save lots of electricity and is exactly what this project needs after running for many years without disproving the conjecture.
ID: 2600 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Slicker
Project administrator

Send message
Joined: 11 Jun 09
Posts: 78
Credit: 943,644,517
RAC: 0
Message 2603 - Posted: 4 May 2020, 21:07:05 UTC

I've never released the source code for this project and do not intend to ever do so. There were people who were cheating which required encrypting the data returned to the server. Releasing the source code would make it easier for people to cheat again.
ID: 2603 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
poppinfresh99

Send message
Joined: 25 Jan 20
Posts: 57
Credit: 1,225,329
RAC: 0
Message 2613 - Posted: 6 May 2020, 14:46:07 UTC - in response to Message 2603.  
Last modified: 6 May 2020, 14:48:18 UTC

Hi Slicker, instead of GitHub, would you consider giving it to individuals?

If I looked it over I could...
- Create a draft for the empty "About Collatz Conjecture" page. Communicating science/math is my jam. So is understanding it.
- Test variations of the code to see if there are improvements to be made, which would only be sent to you.

Believe it or not, I want to help this project, so I would never release the code to the public. If I feel that action should be taken by this project that isn't being taken, I would publicize that, but I wouldn't ruin the project by releasing the source publicly.

Or maybe you could release everything but the encryption aspects?

Are you the only person with access to the code? If so, doesn't that worry you? Scientists never work alone, so maybe you could create a private repository for the code where you could restrict access to certain passwords, IP addresses, or whatever? Or share via email?
ID: 2613 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Bryan

Send message
Joined: 21 Nov 20
Posts: 1
Credit: 0
RAC: 0
Message 2983 - Posted: 21 Nov 2020, 3:56:50 UTC

I too would just like to understand the algorithm and see if there are any tweaks to make it faster.
ID: 2983 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Crunch3r
Volunteer moderator
Volunteer developer

Send message
Joined: 30 Jun 09
Posts: 5
Credit: 7,535,462,606
RAC: 0
Message 3137 - Posted: 6 Feb 2021, 23:07:37 UTC - in response to Message 2600.  

You're not getting the code. Period. There's a reason for that, you've been told why, yet you seems to start bashing the project.
This will stop now.
ID: 3137 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Slicker
Project administrator

Send message
Joined: 11 Jun 09
Posts: 78
Credit: 943,644,517
RAC: 0
Message 3145 - Posted: 8 Feb 2021, 5:20:07 UTC - in response to Message 2983.  

See https://en.wikipedia.org/wiki/Collatz_conjecture . All of the optimizations there have been included in the application. I've had a few people offer optimizations for snippets of the code. What many don't understand is that ideas such as "skip checkpointing and just make the files small enough finish in under 5 minutes" may sound like an obvious solution, right up until you have 70,000 computers asking for 2 days of work when each workunit only takes 10 seconds to run (so that a CPU workunit takes under 5 minutes). That results in millions of workunits a day and unless someone has a super computer and endless bandwidth to donate, it doesn't work. Been there. Done that. MySQL doesn't handle that kind of data well, especially not unless the entire database fits in memory. I've tried different sized workunits and that didn't work since the BOINC scheduler was always out of work for one or the other depending upon which people chose to implement. When competitions, such as the BOINC Pentathalon, begin people hoard workunits to get the credits during the competition. With multi-host verification, the database grows to an extent that it becomes slow as molasses resulting in the server daemons getting backed up for hours. If they get backed up too far, they just create more work. It's a catch 22. No work? Create more. Database is slowed down even more so the insert's are backed up and don't show up so the query times out. So, still no work? Create even more. The database is slowed even more. The next thing you know, you have created a million workunits all of which need verification and only 65K in progress. It will take weeks for them to get validated and in the mean time, people bitch because they haven't gotten credit for the work they've done. THAT's why this project worked very hard to come up with self validating workunits. Anything done via a high speed optimized methiod gets re-checked using TWO much slower non-optimized methods to guarantee the values are correct.

So, as much as I'd love to have the fastest app possible, the reality is that it has to be balanced with the BOINC client, and the way the BOINC server processes the results the way that the open source database can handle the data, and the ability to guarantee that the result is verified even though only one computer processed the result.

I've considered (and even started) porting BOINC to run with SQL Server but that would take several year's effort for a single individual and by the time it was done, it would be years behind the current version.

In reality, solving this mathematical conjecture will do nothing. It won't cure world hunger. It won't change the way kids learn math. It isn't related to any other math conjecture or theories that I know of. For $1200 ( the appox. value of the reward for solving it), if the reward still exists today, no one will get rich (or likely even pay for the power you used to participate in the project) compute a result someone disproves the conjecture, and since the only accepted paper so far is that it is impossible to prove, the only option left is to disprove. No one has done that for decades. In other words, it's a hobby as it has no know applications.

The application is not, nor will ever be open source. If you have questions about how something is done is particular, I'll be happy to answer it and even post code snippets. Due to the number of cheaters, I won't ever allow non-approved optimized apps. But, if someone would like to take a shot at creating a super fast version that is bullet proof, I'll be happy to provide the function input and output requirements. It will still take months to verify all the versions of the app (Win32, Win64, Win32GPU, Win64 GPOU, Linux 32, Linux64, Linux32 GPU, Linux64 GPU, OS X CPU, .you get the idea...)

I don't get paid to do this. I don't get paid to host this. I don't get paid to support this. I don't get paid for the servers that run this or the power to keep them running or the dis space required to back them up. Nor will I. It's a hobby. I won't pay you to do your hobbies, so I don't expect you to pay for mine.

NONE of the contributed apps or optimizations has ever run as well as it did stand alone compared to when run as a sub-process of the BOINC client. There's a fair amount of overhead having a process be controlled and respond to commands from the parent process, especially if the process has child processes (e.g. GPU kernels) that it also has to track and control. That's not to say that any version of the apps can't be improved. I'm sure every one can.

So, ask one question at a time and I will try and answer it. Post a novel and I'll likely ignore it That way I can also respond with "asked and answered" to those who are too lazy to read , have no reading comprehension, or are products of the current school system and only read the first sentence of any document before "Squirrel!" (My 6 year old German Shorthaired Pointer has better attention to detail than many people I know. Just saying...LOL)
ID: 3145 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Arnie

Send message
Joined: 30 Jan 21
Posts: 1
Credit: 1,037,151,785
RAC: 0
Message 3146 - Posted: 8 Feb 2021, 16:03:44 UTC - in response to Message 3145.  

Thanks very much for your explanation. I look forward to reading more answers as they arrive.
Arnie
ID: 3146 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Wedge009

Send message
Joined: 21 Jan 20
Posts: 4
Credit: 5,720,325,616
RAC: 0
Message 3157 - Posted: 10 Feb 2021, 10:36:33 UTC

I brought up some questions/concerns in another thread, but my comment here is specifically for the question of the closed-source nature of the project. On the basis on what has been written here, it seems to me the primary justification for not releasing the source code is the issue of cheating. I assume 'cheating' in this context relates to falsifying results in order to gain an inflated credit score - I fail to see why anyone would bother to such a thing, but I can understand it as a valid concern.

Aside from the benefits of collaboration (especially helpful for small project teams) and verification that releasing source code can bring, couldn't the issue of cheating be mitigated by public-key cryptography or some similar means? Or couldn't there be some sort of separation where only the computational aspects of the code are released and any validation portion remain closed? I understand the desire to minimise cheating, but as one standing on the outside it doesn't sound like a strong reason to me to avoid releasing source code (and there's no disrespect intended in that).
ID: 3157 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 11 Aug 09
Posts: 927
Credit: 24,523,632,110
RAC: 0
Message 3158 - Posted: 10 Feb 2021, 12:34:53 UTC - in response to Message 3157.  

I brought up some questions/concerns in another thread, but my comment here is specifically for the question of the closed-source nature of the project. On the basis on what has been written here, it seems to me the primary justification for not releasing the source code is the issue of cheating. I assume 'cheating' in this context relates to falsifying results in order to gain an inflated credit score - I fail to see why anyone would bother to such a thing, but I can understand it as a valid concern.

Aside from the benefits of collaboration (especially helpful for small project teams) and verification that releasing source code can bring, couldn't the issue of cheating be mitigated by public-key cryptography or some similar means? Or couldn't there be some sort of separation where only the computational aspects of the code are released and any validation portion remain closed? I understand the desire to minimise cheating, but as one standing on the outside it doesn't sound like a strong reason to me to avoid releasing source code (and there's no disrespect intended in that).


A couple years ago there was a guy who was cheating to get more Gridcoins, since Collatz is part of the GridCoin network of Boinc Projects it was open to cheating and then when you throw in the amount of credit given for each task the possibility was to good for someone to pass up. You earn GridCoins by crunching but you earn MORE GridCoins by doing more of it than your fellow crunchers, ie a bigger slice of the pie. The Admin took a couple of months to shut down Collatz and rewrite everything to hopefully prevent another occurrence of cheating, no I never saw the details of how they were cheating and it really doesn't matter as that loophole is now closed anyway.
ID: 3158 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Wedge009

Send message
Joined: 21 Jan 20
Posts: 4
Credit: 5,720,325,616
RAC: 0
Message 3159 - Posted: 10 Feb 2021, 13:08:02 UTC
Last modified: 10 Feb 2021, 13:08:40 UTC

I see, so that explains the motivation. Is this project still part of the GridCoin network? There must surely be way for the validator to guard against such behaviour - one wonders how other projects, GridCoin related or not, avoids falsification of results. A result quorum of more than 1 certainly works towards this, I'm sure.
ID: 3159 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Justin Gregg

Send message
Joined: 18 Jan 21
Posts: 2
Credit: 173,929,741
RAC: 0
Message 3160 - Posted: 10 Feb 2021, 19:23:17 UTC

<preface>I don't have a horse in this race. I run a company that uses and ships both open-source and closed-source software products. I see the value in both. I also don't mean to try to make additional work on this fun hobby.</preface>

One option that might appease both sides in this discussion is to make source available for the core kernels, but not the BOINC interactions or result encryption components. This would appease those interested in "summer reading", and also show the necessary inputs/outputs for people interested in trying their hand at making and testing their own "super fast" and "bullet proof" kernel (as Slicker mentioned as a possibility).
ID: 3160 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Wedge009

Send message
Joined: 21 Jan 20
Posts: 4
Credit: 5,720,325,616
RAC: 0
Message 3163 - Posted: 11 Feb 2021, 5:04:01 UTC

Indeed - I did make a similar suggestion earlier on, obviously it's hard to say how feasible it would be to achieve such a separation without knowing the code in the first place.

I want to point out that this isn't a call for releasing the code just for its own sake, at least in the first instance. While I generally agree having open-source code is more beneficial for everyone, I concede there may be legitimate reasons to keep some things closed. The specific point I'd like to see addressed here is the validity of the science purportedly being run here, or at least steps towards transparency that can help build confidence in the project and give opportunity for subsequent project to build upon the work here. Have the code released could be a small step towards that.
ID: 3163 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 11 Aug 09
Posts: 927
Credit: 24,523,632,110
RAC: 0
Message 3164 - Posted: 11 Feb 2021, 11:42:26 UTC - in response to Message 3159.  

I see, so that explains the motivation. Is this project still part of the GridCoin network? There must surely be way for the validator to guard against such behaviour - one wonders how other projects, GridCoin related or not, avoids falsification of results. A result quorum of more than 1 certainly works towards this, I'm sure.


Yes Collatz is still one of the the Boinc Projects that allow GridCoins to be earned by crunching for them, but Collatz itself does not control anything about GridCoins.
ID: 3164 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
CodeRedDewd

Send message
Joined: 27 Mar 10
Posts: 1
Credit: 345,293,643
RAC: 0
Message 3169 - Posted: 18 Feb 2021, 6:43:30 UTC - in response to Message 3164.  

I installed the Gridcoin wallet and have an address to send the Gridcoin to, but I don't know where to receive them or how to set it up. I have not seen a Gridcoin anything anywhere. Anyone have theirs wet up and can help?
ID: 3169 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile mikey
Avatar

Send message
Joined: 11 Aug 09
Posts: 927
Credit: 24,523,632,110
RAC: 0
Message 3170 - Posted: 18 Feb 2021, 11:37:59 UTC - in response to Message 3169.  

I installed the Gridcoin wallet and have an address to send the Gridcoin to, but I don't know where to receive them or how to set it up. I have not seen a Gridcoin anything anywhere. Anyone have theirs wet up and can help?


Did you put the address into your wallet and join a Pool or the GC Team and send a 'beacon' to the GC network telling it you are ready to start earning GC's yet?
Until you have some GC's you can't earn any GC's either, so you will need someone to send you some to start the process. Thereafter it depends on how many you have on how often you get sent your daily share, the closer to 100k GC's you have the closer to once a day you get 'staked'. You do NOT get less GC's if you only get 'staked' once a week or even once a month, you get exactly the same amount the Server keeps track of them until it sends you your share. Your share depends on your 'magnitude' which is how much you contribute to the group of GC Projects, not every Boinc Project is a part of it as some have opted out or not been contacted because they are so small. Being a part of the GC group can mean a large increase of users and not all projects want that while others chose not to for their own reasons.
ID: 3170 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Science : Open source


©2022 Jon Sonntag; All rights reserved