Migrating MusicXML import to pull parser

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Migrating MusicXML import to pull parser

Leon Vinken
For your information, I am currently working on migrating the MusicXML parser from the current DOM parser to the (new) pull parser, which is also used for reading MuseScores .mscx files.

As this is a significant effort (the MusicXML importer is about 8500 lines of code), it is most efficient to do it while the MuseScore core is quite stable (which is now, before we start work on new things for the post 2.0 release). Due to the obvious risk, this should not be included in 2.0 anymore. It is my intention to be ready for commit soon after 2.0 is done.

Note that I consider neither of the current open MusicXML importer issue a showstopper and do not intend to work on these until the pull parser migration is done. If anyone disagrees please let me know.

Finally, consider this a request to try to limit MusicXML importer changes to a minimum until the migration is done, or consult me about how to do this without causing a significant merge effort.

Regards, Leon.
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Marc Sabatella
There is one thing I would like to see changed in MusicXML import (and export) before 2.0 if possible, and that is open/atonal key signatures - http://musescore.org/en/node/46986.  It should be an extremely easy change, and I'm happy to do it myself, except I don't know what the MusicXML *should* look like for these key signatures.

On Tue, Feb 10, 2015 at 2:28 PM, Leon Vinken <[hidden email]> wrote:
For your information, I am currently working on migrating the MusicXML parser
from the current DOM parser to the (new) pull parser, which is also used for
reading MuseScores .mscx files.

As this is a significant effort (the MusicXML importer is about 8500 lines
of code), it is most efficient to do it while the MuseScore core is quite
stable (which is now, before we start work on new things for the post 2.0
release). Due to the obvious risk, this should not be included in 2.0
anymore. It is my intention to be ready for commit soon after 2.0 is done.

Note that I consider neither of the current open MusicXML importer issue a
showstopper and do not intend to work on these until the pull parser
migration is done. If anyone disagrees please let me know.

Finally, consider this a request to try to limit MusicXML importer changes
to a minimum until the migration is done, or consult me about how to do this
without causing a significant merge effort.

Regards, Leon.




--
View this message in context: http://dev-list.musescore.org/Migrating-MusicXML-import-to-pull-parser-tp7579106.html
Sent from the MuseScore Developer mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer



--
Marc Sabatella
[hidden email]

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

lasconic
Administrator
Hi Leon,

Thank you for the notice! I agree that it's the right time to do this.

There is one PR that you might want to consider before starting the move to the pull parser.
I didn't merge it yet because I wanted your opinion about it.

lasconic


2015-02-10 23:16 GMT+01:00 Marc Sabatella <[hidden email]>:
There is one thing I would like to see changed in MusicXML import (and export) before 2.0 if possible, and that is open/atonal key signatures - http://musescore.org/en/node/46986.  It should be an extremely easy change, and I'm happy to do it myself, except I don't know what the MusicXML *should* look like for these key signatures.

On Tue, Feb 10, 2015 at 2:28 PM, Leon Vinken <[hidden email]> wrote:
For your information, I am currently working on migrating the MusicXML parser
from the current DOM parser to the (new) pull parser, which is also used for
reading MuseScores .mscx files.

As this is a significant effort (the MusicXML importer is about 8500 lines
of code), it is most efficient to do it while the MuseScore core is quite
stable (which is now, before we start work on new things for the post 2.0
release). Due to the obvious risk, this should not be included in 2.0
anymore. It is my intention to be ready for commit soon after 2.0 is done.

Note that I consider neither of the current open MusicXML importer issue a
showstopper and do not intend to work on these until the pull parser
migration is done. If anyone disagrees please let me know.

Finally, consider this a request to try to limit MusicXML importer changes
to a minimum until the migration is done, or consult me about how to do this
without causing a significant merge effort.

Regards, Leon.




--
View this message in context: http://dev-list.musescore.org/Migrating-MusicXML-import-to-pull-parser-tp7579106.html
Sent from the MuseScore Developer mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer



--
Marc Sabatella
[hidden email]

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer



------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

David Bolton-2
In reply to this post by Marc Sabatella
Marc,

I posted on the bug report with the export from Finale 2014.

David



On 2/10/2015 4:16 PM, Marc Sabatella wrote:
> There is one thing I would like to see changed in MusicXML import (and
> export) before 2.0 if possible, and that is open/atonal key signatures
> - http://musescore.org/en/node/46986. It should be an extremely easy
> change, and I'm happy to do it myself, except I don't know what the
> MusicXML *should* look like for these key signatures.
>



------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Marc Sabatella
i responded there.  I'm thinking we aren't talking about the same thing.

On Wed, Feb 11, 2015 at 6:52 PM, David Bolton <[hidden email]> wrote:
Marc,

I posted on the bug report with the export from Finale 2014.

David



On 2/10/2015 4:16 PM, Marc Sabatella wrote:
> There is one thing I would like to see changed in MusicXML import (and
> export) before 2.0 if possible, and that is open/atonal key signatures
> - http://musescore.org/en/node/46986. It should be an extremely easy
> change, and I'm happy to do it myself, except I don't know what the
> MusicXML *should* look like for these key signatures.
>



------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer



--
Marc Sabatella
[hidden email]

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Leon Vinken
This post was updated on .
In reply to this post by lasconic
@lasconic,

I will try to evaluate the pull request during the weekend.

Leon.
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Leon Vinken
Quick status update:

I am making reasonable progress and think the main / most difficult parts are done. The new pull parser now contains about 6400 lines of code compared to the DOM parsers 7200 lines. About half the auto testers test cases pass (46 out of 95).

Given the amount of work remaining and the time left, I do not expect to be able to finish the pull parser before March 24th. Current plan is to focus on finishing the "most used" features and postpone some low risk "less used" features such as harmonies and tablature. I think these can safely be added later.

Nevertheless, I still think it is necessary to merge the status as is immediately after release of 2.0. The simple reason being that once major changes are made to the core, I will never be able to keep up and basically all the effort spent on the pull parser will be wasted. As this currently amounts to about two full weeks of effort, I would be highly disappointed if that would happen.

A question to the core developers about strategy: in my branch the pull parser is a compile time switch. Options are to leave the switch in (both the DOM and pull parser source code are in the release, more migration effort for core changes, easy fallback in case the pull parser fails) or to simply remove the switch and the DOM parser, leaving only the pull parser. I can easily do both, what would your preference be ?

Regards, Leon.
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

lasconic
Administrator
Great to hear about your progress !

About your question. It's mainly a question of confidence and test coverage. If when you are ready to merge, or after 2.0 is released, you are confident that the pull parser is close to on par with the DOM one, then I would merge the pull parser only. If not, I would merge both with the compile time switch. Up to you to find the sweet spot between the burden on maintaining two parsers and the burden of keeping up to date with the changes in core. I hope it answers your question.

lasconic

2015-03-10 22:51 GMT+01:00 Leon Vinken <[hidden email]>:
Quick status update:

I am making reasonable progress and think the main / most difficult parts
are done. The new pull parser now contains about 6400 lines of code compared
to the DOM parsers 7200 lines. About half the auto testers test cases pass
(46 out of 95).

Given the amount of work remaining and the time left, I do not expect to be
able to finish the pull parser before March 24th. Current plan is to focus
on finishing the "most used" features and postpone some low risk "less used"
features such as harmonies and tablature. I think these can safely be added
later.

Nevertheless, I still think it is necessary to merge the status as is
immediately after release of 2.0. The simple reason being that once major
changes are made to the core, I will never be able to keep up and basically
all the effort spent on the pull parser will be wasted. As this currently
amounts to about two full weeks of effort, I would be highly disappointed if
that would happen.

A question to the core developers about strategy: in my branch the pull
parser is a compile time switch. Options are to leave the switch in (both
the DOM and pull parser source code are in the release, more migration
effort for core changes, easy fallback in case the pull parser fails) or to
simply remove the switch and the DOM parser, leaving only the pull parser. I
can easily do both, what would your preference be ?

Regards, Leon.



--
View this message in context: http://dev-list.musescore.org/Migrating-MusicXML-import-to-pull-parser-tp7579106p7579132.html
Sent from the MuseScore Developer mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Mscore-developer mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mscore-developer
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Leon Vinken
Pull request 1911.
Regards, Leon.
Reply | Threaded
Open this post in threaded view
|

Re: Migrating MusicXML import to pull parser

Leon Vinken
@lasconic,

if there is anything I need to do to get the pull parser merged, please let me know what it is.

Any idea when you want to merge it ?

Regards, Leon.