OCR for Musecscore?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

OCR for Musecscore?

Andrei Borovsky
Hi
Being a programmer and a fan of Musescore I'd like to offer my
assistance in some particular field. Does Musescore community need an
OCR capability for the software? I have some experience in development
some general propose OCR soft (YAGF, Cuneiform), so I could contribute
to printed score recognition development for Musescore if sufficient
number of people is interested.
Regards,
Andrei


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OCR for Musecscore?

lasconic
Administrator
Hi,

I assume you are talking about OMR or Music OCR. See https://en.wikipedia.org/wiki/Music_OCR.

As far as I understand, OMR is quite different from OCR. Of course, there are similarities, but music is quite different from text.

Of course, everybody wants to take a photo of a sheet music and have a perfect digital version to modify, play, transpose etc... so there is no question about the interest of such a feature. There are several endeavors in MuseScore and the free software community regarding OMR. 

First, there is Audiveris http://audiveris.org The V4.3 is the last public version and work is going on privately for Audiveris 5. I have no idea what the status is. MuseScore.com provides a PDF to MuseScore service as best effort thanks to Audiveris 4. https://musescore.com/import 
Audiveris is in the work for years. I fear that starting such an effort from scratch would be very hard.

Several years ago, MuseScore's lead developer Werner Schweer started another effort which does use some OMR techniques but has a different approach. He wanted to make it easier to transcribe existing sheet music manually. So he implemented staff, barline detection and a UI to sync images with MuseScore sheet music. It looked like this http://www.screenr.com/g1Bs Unfortunately, this feature didn't make it in MuseScore 2.0. 

During the last 6-8 months, Liang Chen http://homes.soic.indiana.edu/chen348/ revived this code and upgrade it, so this feature is present again in the current master (and in the nightly builds https://musescore.org/en/download#Nightly-versions). It's still a work in progress and Liang is still working on it. Some things are better than the previous implementation, others are missing (for example Werner used Tesseract to extract titles etc..., and this has not been added back). 
If you want to help, take a look to the existing code, build MuseScore and pass by IRC #musescore on freenode.net to discuss the current state and future development. You can also see the README in the OMR directory https://github.com/musescore/MuseScore/tree/master/omr#updates-on-optical-music-recognition

I didn't touch about live handwritten music recognition because your email talked about printed score recognition but that's also close from OCR and of course, some people want this, in particular on touch based devices and with the buzz about proprietary software (Staffpad, notateme,...) . As far as I know, there is no free software effort in this direction (If anyone knows about such a project, please share!).

lasconic




2016-05-29 17:03 GMT+02:00 Andrei Borovsky <[hidden email]>:
Hi
Being a programmer and a fan of Musescore I'd like to offer my
assistance in some particular field. Does Musescore community need an
OCR capability for the software? I have some experience in development
some general propose OCR soft (YAGF, Cuneiform), so I could contribute
to printed score recognition development for Musescore if sufficient
number of people is interested.
Regards,
Andrei


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer


------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OCR for Musecscore?

Liang Chen
In reply to this post by Andrei Borovsky
Hi,

I was working on the OMR in musescore in the last several months. What I was mainly focused on was pdf to image conversion using poppler and automatic system identification. After system identification (separate the score into systems, in each system there'll be measures sharing common bar lines), we'll be able to generate a MuseScore "skeleton" and align it to the omr view. This is to facilitate users' transcription process but didn't cover the core part of OMR: symbol recognition and music interpretation.

If you're interested we can move on to add symbol recognition such as clef/key signatures and notes to the module. Clef and key signatures have the same one-dimensional layout as text but the notes are much trickier due to polyphony. The other possibility is to open the OCR flag and do text recognition for title identification or instrumentation recognition. That will be more closely relevant to OCR. Either direction will be interesting exploration and be helpful.

Thanks,
Liang
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OCR for Musecscore?

Andrei Borovsky
In reply to this post by Andrei Borovsky
Hi Liang,

Thank you for you letter. If I got you right you are working with well
formed pdf images. My skills and interest lie more in the field of
recognizing scanned pages (skewed, distorted, with sudden
brightness/contrast splashes). I think I could build a module that would
recognize such scans and turn the music information into some
intelligible form (I'm not well acquainted with MS score internal
representation yet). Yes it would be nice to work on recognizing clefs
and notes and the like. BTW one thing that I saw in one of the systems -
counting score timing as a sort of control sum to check that everything
has been recognized well. For text recognition it woulld be great to use
libtesseract as it has all the things in place. BTW do you know anything
about the Audiveris project? On thier site they claim MS integration as
their goal.

Regards,
Andrei



31.05.16 18:12, Liang Chen пишет:

> Hello Andrei,
>
> I was working on the OMR in musescore in the last several months. What I was
> mainly focused on was pdf to image conversion using poppler and automatic
> system identification. After system identification (separate the score into
> systems, in each system there'll be measures sharing common bar lines),
> we'll be able to generate a MuseScore "skeleton" and align it to the omr
> view. This is to facilitate users' transcription process but didn't cover
> the core part of OMR: symbol recognition and music interpretation.
>
> If you're interested we can move on to add symbol recognition such as
> clef/key signatures and notes to the module. Clef and key signatures have
> the same one-dimensional layout as text but the notes are much trickier due
> to polyphony. The other possibility is to open the OCR flag and do text
> recognition for title identification or instrumentation recognition. That
> will be more closely relevant to OCR. Either direction will be interesting
> exploration and be helpful.
>
> Thanks,
> Liang
>
>
>
> --
> View this message in context: http://dev-list.musescore.org/OCR-for-Musecscore-tp7579927p7579929.html
> Sent from the MuseScore Developer mailing list archive at Nabble.com.
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
> _______________________________________________
> Mscore-developer mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/mscore-developer
>



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: OCR for Musecscore?

Liang Chen
Hi Andrei

I'm not sure what you exactly meant by "well-formed" pdf. In fact, we already have some deskewing function in MuseScore's lightweight omr, so we're able to handle some moderate distortion of the page. We also take scanned scores as default input; the test data was acquired from the online score repository IMSLP (see some here: https://github.com/liang-chen/MuseScore/tree/omr_dev_clean/omr/data). But if we can enhance or restore the low-quality pdf's in an efficient way, I believe that'll benefit the later recognition process.

Regarding OCR, the previous version developed by Werner Schweer had already used tesseract. We might need to revisit this part and enable it for the current program. If you're more interested in the current status, just take a look at this folder: https://github.com/musescore/MuseScore/tree/master/omr.

I don't know in detail how Audiveris works, but I believe the future development of OMR should depend more on the recent breakthroughs of computer vision (graphical models, deep learning etc.) rather than heuristic-based approach (ad hoc segmentation, pixel projections etc). Traditional OMR used a lot of heuristics and the performance was quite unreliable.

Liang

Loading...