Lazarus Examples Window

This wiki page now a reference for the new Example Window that exists, at present, in lazarus-main but not a release version. Its based on this forum thread - https://forum.lazarus.freepascal.org/index.php/topic,57680.0.html and is by no means finalized.

What should the user experience be like ?

• A user should be able to choose to see content appropriate to their experience.
• Be able to see a summary of what an Example is about before opening it.
• Be able to open an Example, have a play with it, make a few changes and recompile to see what happens. Maybe roll back to the original Example if they make a full mess of it.
• Examples should be generally short, contain appropriate, relevant code focused on one topic each. But there will be many exceptions to that.
• If a user installs a third party package, eg via OPM or manually, then any examples it contains should be treated in the same manner.
• Be useful to allow a user to browse through example content while working on a real project, copy and past a snipit as required.

How it Works

The Lazarus IDE Examples Window shows all (compliant) examples in a list, filterable by a category and searchable by name or keyword. As of Lazarus Main April 7th, it finds examples shipped with the Lazarus Source but few if any other packages. To allow the IDE to find and use an example in a package the following is needed -

• The the example or examples in the package must have a suitable metadata file, its a simple JSON file with only a few fields.
• The example must be self contained in its own directory (and any directories below it). A shared file, or one in a specific location in the package tree will NOT work.
• The package must be installed in the IDE using either OPM or a manual install and the IDE can find it.

The Metadata File

All examples that appear in the Example Window have a json metadata file with an extension ".ex-meta" in the top level directory of the example (not the package). Without a valid metadata file, the example will not be listed in the Example Window. An individual example project file would look like this -

{ 
"Laz_Hello" : {
    "Category" : "Beginner",
    "Keywords" : ["Lazarus", "Hello World", "TButton", "TMemo"],
    "Description" : "This might be your first Lazarus project, its the traditional Hello World.  Two buttons have been dropped on the form, renamed and their captions set. A memo was then added and each button was double clicked and what they are expected to do was set."
}

In this case, the example project files would be in a subdirectory called laz_hello, same spelling but all lowercase consistent with Lazarus.

The three fields are free form text, there is no dictionary so, you are free to introduce both new Categories and new Key Words. Its desirable that we do not have too many Categories so, perhaps consider posting a message on the forum before using a new one ? At present, the following Categories are in use Beginner; General; TAChart; DBase; LazReport.

There is a rough and ready GUI app to make, edit and importantly, validate metadata files available at https://github.com/davidbannon/ExampleMetaData .

A standalone example ?

When user opens an example project (from the Example Window) it has been copied to a work area (in the Lazarus PCP) so if the example depends on other files from the package source tree, it won't find them. So, ensure all needed files are in or under the example top level directory. Sometimes, this may lead to duplication of files in the package tree but it is necessary for several reasons, firstly, Linux users using distro versions of Lazarus have all their examples in read only diskspace, secondly, its important that we can restore an example if the user messes it up badly.

Can the IDE find the example ?

This is a question not completely answered at this stage. An example project, part of an installed package will be found if - TBC

Notes

The user can view the project content and, if they choose, build the project. Later, if the user users the Example Window to again open that project, they will be asked if they want to refresh it, doing so will erase any false edits they may have made.

The Example Window will also scan the PCP (or location where On-Line Package manager puts packages) so, any packages installed via the OPM that have Examples and a valid metadata file will be treated the same way.

Important to note -

• Example Projects will not be displayed in the Examples Window unless they have a valid metadata file. An error is dropped to the console if a metadata file with bad json is encountered.
• The original copy of the Example is no altered when the Example Window is used to open an Example. So, play away !
• New examples can, potentially be added via the normal Gitlab pull request system.
• Examples demonstrating aspects of OPM packages are probably best added to that that package rather than to Lazarus itself.

See also

• https://forum.lazarus.freepascal.org/index.php/topic,57680.0.html
• https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/37509
• https://gitlab.com/dbannon/laz_examples - this is just a temp home.
• https://gitlab.com/dbannon/laz_examples/-/tree/main/Utility/ExScanner - a tool for manipulating Example Projects in and around the Lazarus Src Tree. Probably past its use by date. Also has a framework to 'exercise' the Lazarus Examples Window in a stand alone, easier to manage environment.
• https://github.com/davidbannon/ExampleMetaData - a very simple editor to make and validate the Example Meta Data files.

Legacy Information

Progress Report Dating back to Development Stages

March 8th, 2022

• A working prototype can now be found at https://gitlab.com/dbannon/lazarus/-/tree/newexamples - this uses Examples in the Source Code, its not On-Line.
• You should git clone or download https://gitlab.com/dbannon/lazarus/-/archive/newexamples/lazarus-newexamples.zip - that will get you an almost current trunk Lazarus with the 'old' Examples Window replaced with the new one and all the examples in ~/examples visible, usable and working.

• Compile the above with the usual "make bigide"
• Start it up, remember to set an alternative config path so not to overwrite your usual Lazarus.
• in the IDE, go to Package -> Open Package File (.lpk) and browse to where you downloaded to, components/exampleswindow/exampleprojects.lpk and open that, click Use->Install and let i rebuild as usual.
• In use, there is a menu entry, Tools->Example Projects or in the startup mode with a project loaded, the ProjectWizard has an Example Projects button.

• Most if not all of the viable examples in Laz-Dir/examples have now been categorized and have a metadata file. Some 104 projects made it through.

Still to do -

• Process the remaining Examples in the Lazarus Source, about 200 ?
• Encourage third party package maintainers to add a metadata file if they have examples.

Why is there a problem ?

The existing Examples system has issues, perhaps in that it has not keep up Lazarus's own rapid development. There are a number of issues identified (in no particular order) -

• Linux Lazarus installs based on (distro) Packages have the Examples in read only space.
• Most Examples do not have any indication of the topic covered other than file/project name.
• Examples don't have a category system identifying who target audience is, beginner to advanced user.
• Many examples are outdated, many don't, for example use the Lazarus Form Designer confusing new users who expect to see Object Inspector content.
• The system is limited to only those examples shipped in the main Lazarus distribution, cannot cover eg Examples applicable to OPM.

Methodology

There appears to be two broad models available, each with their own advantages and disadvantages. Choosing one or the other model and determining what our metadata file looks like is the next phase.

Its likely that the metadata design is substantially the same either model.

Both Example Model will require -

• An extensive review of all existing Examples is required (for both models).
• Additional of a metadata file.
• A call for more Examples.
• A means to scan for 'other' Examples, such as ones in OPM packages.

Examples Remain in Distribution.

This is the KISS solution, it involves the least structural change but may not necessarily involve less work. As well as reviewing and adding meta data, we may wish to re-organize the location of packages.

It would make sense to still move a selected project from its location in the SRC tree to (eg) Lazarus config area to to -

• Solve the Linux Read Only problem and
• allow a user who has edited an example project excessively to "roll back" and start again.
• continuity if the user updates their Lazarus.

The Process is probably to add the meta data to each existing project over time and eventually, when that is all done, alter the existing Lazarus Examples Window to use the new model. The metadata file will aide the existing Lazarus Example Window but will do no harm either.

The down sides of this approach are -

• An end user cannot browse the Examples (online). They will need to close a project they are working on, find the relevant Example and open it in Lazarus.
• The changes that must happen to the existing Examples will need extensive Lazarus Developer involvement. Either one very scary diff or lots of somewhat smaller ones.

Examples move on-line

Possibly a simpler change because much of it is implemented externally, at some stage the Examples Window will need to be changed in the Lazarus Distro but lots of building, adjusting of existing Examples, testing can happen without annoying Lazarus Developers.

Prototype code to download files, index projects and build a master metadata file already exists.

A very rough and ready example example repository has been established at https://gitlab.com/dbannon/laz_examples

The process would be to complete the metadata and testing of all Examples. Next the existing gitlab tree at under dbannon would move to being under the FPC/Lazarus, finally, the Lazarus Examples Window would change to point to the Examples in the official gitlab tree. At some stage, the existing Examples would be removed from the Lazarus SRC, perhaps leaving a small representative number.

The downsides of this approach -

• Its a drastic structural change.
• And end user requires network access to download an Example.

Using the Demo App

Firstly, you can browse the repository, just looking for snipits of code, you can download a particular Example directory as a zip file and use it locally. This is perhaps, for many users, easier than closing their existing project, opening up the example, seeing what they are looking for and going back to their real project and continuing working. But you need online access....

An application now exists that makes creating and managing the metadata files. We'll call it EMT, Example Management Tool, https://gitlab.com/dbannon/laz_examples/-/tree/main/Utility/ExScanner . It will copy a project's files into a ready to push git directory. The process is -

• Check the default directories in EMT, they are set to suit me, probably not you !

• In EMT, press the Set button and point to the directory containing the example you plan to test and process. Important you do this before compiling the project otherwise you will be trying to add compiled binary files.

• Open that project in Lazarus, make sure it works, does what it appears to claim to do and if all OK, fill in the meta data in EMT, if the project has a readme.txt file, use at least some of its content. Click Save.

EMT will also allow you the directly edit a metadata file and save it back where it came from. And it incorporates a draft Lazarus Examples Windows, a prototype of what may go into Lazarus eventually.

In the event that we re-introduce the On Line Examples model, the master.ex-meta file, regenerated frequently and downloadable from the repo to be cached locally would look like this -

{ 
"LastUpDate" : "2021-10-19T11:44:41+11:00",

"Beginner/laz_hello" : {
    "Name" : "Laz_Hello",
    "Category" : "Beginner",
    "Keywords" : ["Lazarus", "Hello World", "TButton", "TMemo"],
    "Description" : "This might be your first Lazarus project, its the traditional Hello World.  Two buttons have been dropped on the form, renamed and their captions set. A memo was then added and each button was double clicked and what they are expected to do was set.",

"Components/listview" : {
    "Name" : "ListView",
    "Category" : "Components",
    "Keywords" : ["TListView", "grid", "needs work"],
    "Description" : "A project that demonstrates some of a TListView's capabilities. Not so much a \"how to use\" as an exerciser of capabilities. Note, entering an invalid column number triggers a crash."

}

Note the date in ISO format, overall, the file is still reasonably human readable. And now, of course, its easily extendable. We could add a format version but I don't consider it necessary.