Comments on: Presentation Summary: “MPTStore: Implementing a fast, scalable, and stable RDBMS-backed triplestore for Fedora and the NSDL” http://dltj.org/article/fedora-mptstore/ We're Disrupted, We're Librarians, and We're Not Going to Take It Anymore Wed, 08 Feb 2012 17:48:39 +0000 hourly 1 By: More Newshttp://dltj.org/article/fedora-mptstore/comment-page-1/#comment-13541 More News Thu, 29 Mar 2007 18:14:34 +0000 http://dltj.org/2007/01/fedora-mptstore/#comment-13541 <!--%kramer-ref-pre%-->[...] MPTStore Presentation Summary on MPTStore. A summary of an interesting approach by the Fedora guys to storing lots of triples, fast.The real motivation behind experimenting with a new triplestore, however, was the NSDL use case. The National Science Digital Library5 (NSDL) is a moderately large repository (4.7 million objects, 250 million triples) with a lot of write activity (driven by periodic OAI harvests; primarily mixed ingests and datastream modifications). The NSDL data model also includes existential/referential integrity constraints that must be enforced. Querying the RI to determine correct repository state proved to be difficult: Kowari is aggressively buffering triple, sometimes on the order of seconds, before writing them to disk. Flushing the buffer after every write is also computationally expensive (hence the drive to use buffers in the first place).Based on this observation, their solution, called “Mapped Predicate Tables,” creates a table for every predicate in the triplestore. This has several advantages: a low computational cost for triple adds and deletes, queries for known predicates are fast, complex queries benefit from the relatively mature RDBMS planner having finer-granularity statistics and query plans, and flexible data partitioning to help address scalability. This solution comes with several disadvantages, however: one needs to manage predicate to table mapping, complex queries crossing many predicates require more effort to formulate, and with a naive approach simple unbound queries scale linearly with the number of predicates.They achieved basically the same performance either asynchronous or synchronous modification.The project is available on Sourceforge, including slides and javadoc (which has a similar design to JRDF except no blank nodes).Labels: rdf, triple store [...]<!--%kramer-ref-post%--> Kramer auto Pingback[...] MPTStore Presentation Summary on MPTStore. A summary of an interesting approach by the Fedora guys to storing lots of triples, fast.The real motivation behind experimenting with a new triplestore, however, was the NSDL use case. The National Science Digital Library5 (NSDL) is a moderately large repository (4.7 million objects, 250 million triples) with a lot of write activity (driven by periodic OAI harvests; primarily mixed ingests and datastream modifications). The NSDL data model also includes existential/referential integrity constraints that must be enforced. Querying the RI to determine correct repository state proved to be difficult: Kowari is aggressively buffering triple, sometimes on the order of seconds, before writing them to disk. Flushing the buffer after every write is also computationally expensive (hence the drive to use buffers in the first place).Based on this observation, their solution, called “Mapped Predicate Tables,” creates a table for every predicate in the triplestore. This has several advantages: a low computational cost for triple adds and deletes, queries for known predicates are fast, complex queries benefit from the relatively mature RDBMS planner having finer-granularity statistics and query plans, and flexible data partitioning to help address scalability. This solution comes with several disadvantages, however: one needs to manage predicate to table mapping, complex queries crossing many predicates require more effort to formulate, and with a naive approach simple unbound queries scale linearly with the number of predicates.They achieved basically the same performance either asynchronous or synchronous modification.The project is available on Sourceforge, including slides and javadoc (which has a similar design to JRDF except no blank nodes).Labels: rdf, triple store [...]

]]> By: the jesterhttp://dltj.org/article/fedora-mptstore/comment-page-1/#comment-13536 the jester Thu, 29 Mar 2007 15:55:32 +0000 http://dltj.org/2007/01/fedora-mptstore/#comment-13536 Kingsley —In this case, I'm just the reporter of someone else's work. I don't know if the Fedora core developers or the NSDL development team considered Virtuoso, but I will pass along your comment to them in case you have not contacted them directly. Kingsley —

In this case, I’m just the reporter of someone else’s work. I don’t know if the Fedora core developers or the NSDL development team considered Virtuoso, but I will pass along your comment to them in case you have not contacted them directly.

]]> By: Kingsley Idehenhttp://dltj.org/article/fedora-mptstore/comment-page-1/#comment-13534 Kingsley Idehen Thu, 29 Mar 2007 15:09:02 +0000 http://dltj.org/2007/01/fedora-mptstore/#comment-13534 Hi There,Would you be happy to repeat these experiments using Virtuoso (see: http://sourceforge.net/projects/virtuoso ).Note: Virtuoso is a High-Performance DBMS for both SQL and RDF (in-built Quad Store). It also has its own in-built Full Text Indexing. It is optimized at all levels. We also control the SQL compiler so we are able to optimize specifically for SPARQL inside the SQL Compiler.Virtuoso current hosts: http://dbpedia.org amongst other large Triple Store projects.Let's do some testing etc.. Hi There,

Would you be happy to repeat these experiments using Virtuoso (see: http://sourceforge.net/projects/virtuoso ).

Note: Virtuoso is a High-Performance DBMS for both SQL and RDF (in-built Quad Store). It also has its own in-built Full Text Indexing. It is optimized at all levels. We also control the SQL compiler so we are able to optimize specifically for SPARQL inside the SQL Compiler.

Virtuoso current hosts: http://dbpedia.org amongst other large Triple Store projects.

Let’s do some testing etc..

]]> By: NSDL Road Reports » Blog Archive » NSDL Presentations at Open Repositories 2007http://dltj.org/article/fedora-mptstore/comment-page-1/#comment-11958 NSDL Road Reports » Blog Archive » NSDL Presentations at Open Repositories 2007 Thu, 01 Feb 2007 19:32:50 +0000 http://dltj.org/2007/01/fedora-mptstore/#comment-11958 [...] Wilper, Birkland, MPT Store: A Fast, Scalable, and Stable Resource Index http://dltj.org/2007/01/fedora-mptstore/ [...] [...] Wilper, Birkland, MPT Store: A Fast, Scalable, and Stable Resource Index http://dltj.org/2007/01/fedora-mptstore/ [...]

]]>