Ideas and discussion for Regnus 5.5

[Ryan]

Well, it's not been very long since the introduction of the new 5.4 specification, but this thread is to begin discussion regarding new feature requests for a future 5.5 specification, including some further thoughts on ideas which were not implemented in the previous update. Please do feel welcome to discuss and post any further ideas or thoughts below.

1) A "DRAFT" qualifier adding "include" functionality (not to be supported in Regnus 5.5)

A suggestion originally mooted for the 5.4 specification is some form of "include" functionality, which would work in a similar way to other programming and scripting languages, in that it would cause an external script to be included in the current script and treated as a single script.

It is already possible, of course, to use the $ RTC to reference content in external scripts, and this already allows for all of the functionality which an include function would provide. However, the subtle difference is that using a reference creates and additional recursion and returns fully processed parsed content from an external script, as the two individual scripts are effectively "blind" to each other's contents. An include function would actually cause the external script to be loaded into memory as if it was part of the current script, meaning that its groups would be accessible in all the usual ways, without having to create handler labels in the second script to grant access to its content.

The proposed syntax for this function would be:

Code: Select all

DRAFT path/filename.rsf

This would be relatively easy to implement, but might not be desirable as it essentially duplicates (though in a simplified fashion) existing functionality, and also, would potentially cause issues as the included script would not necessarily be complete and valid, and would cause duplicate metadata qualifiers if it were to be a complete valid script. External scripts in this case would be more likely script "fragments" containing only groups of entries with no labels, for instance, so it may be necessary for simplicity of debugging to include new metadata and specify that a particular file is deliberately a fragment rather than a complete script.

Another issue that would be important to consider would be how this new qualifier would behave depending on where placed within a script. One implementation would be that, no matter where placed, the qualifier would append the given fragment to the end of the script, meaning that all groups within the fragment would need to be complete. Another implementation would be to allow fragment files to be inserted directly into certain points within a script, allowing (for example) additional entries to be inserted directly into groups. This would be powerful, but would also be more difficult to implement. A third option would be to allow selective importing of data from an external script. For example, only the external script's groups might be imported (or even specific groups?), with all labels stripped, allowing lists of data to be imported from complete external scripts and thus also solving the issue of the functionality requiring the creation of "fragment" scripts as discussed above.

Another important consideration regarding this proposed functionality is that it would almost certainly add additional loading time to scripts which don't even use drafting, and would add a level of complexity to the language which might be undesirable given that it adds a minor convenience for some authors, but no new functionality.

2) A new type of expression to allow inline entry selection! (implemented!)

Currently, Regnus has three types of expression: the group expression, the literal expression, and the slot expression, each of which can be used in any reference to return text in a particular way. A literal expression (enclosed in parantheses) returns the given text or phrase, parsed, allowing nested references; the slot expression (enclosed in square brackets) returns the content stored in the given slot number (specified as a numeric value, or as a named variable defined by a REFER qualifier); and the group expression (unenclosed) returns an entry selected from the specified group.

This last type of expression is very powerful, and is essentially the core of Regnus' capabilities, but is sometimes heavy-handed for simple selections. When using a reference to choose between only a few short words, the additional work of setting up a separate group to contain these sometimes feels a little arduous for a minor level of randomisation which will only be used once in a script. For example, see the following script:

Code: Select all

GROUP FirstGroup
ENTRY <Example>I have ##SecondGroup; bananas.

GROUP SecondGroup
ENTRY some
ENTRY no

The suggested new expression type would allow the two entries from "SecondGroup" to be included inline in the labelled entry. For example:

Code: Select all

GROUP FirstGroup
ENTRY <Example>I have ##{some¦no}; bananas.

And of course, as this is an expression type, it would be usable with all other RTCs. For example:

Code: Select all

GROUP FirstGroup
ENTRY <Example>I have #>(0):{some¦no}; bananas; ##[0]; bananas have I!

Consideration should be given when implementing this feature as to whether the contained text should itself be parsed; and if so, how this should be allowed, and what additional complications or issues might arise. For example:

Code: Select all

GROUP FirstGroup
ENTRY <Example>I have ##{##SecondGroup;¦##ThirdGroup;¦##FourthGroup;}; bananas.

GROUP SecondGroup
ENTRY some
ENTRY no

GROUP ThirdGroup
ENTRY excellent
ENTRY enormous
ENTRY fantastic

GROUP FourthGroup
ENTRY yellow
ENTRY green
ENTRY pink

Or even:

Code: Select all

GROUP FirstGroup
ENTRY <Example>I have ##{##SecondGroup; ##ThirdGroup;¦##SecondGroup; ##FourthGroup;}; bananas.

GROUP SecondGroup
ENTRY some
ENTRY no

GROUP ThirdGroup
ENTRY excellent
ENTRY enormous
ENTRY fantastic

GROUP FourthGroup
ENTRY yellow
ENTRY green
ENTRY pink

One question is whether an entirely new expression type would be the best approach (as above), or whether it would be more elegant to add this functionality to the existing literal expression type instead? In this case, literal expressions would be treated in the same way as they are currently, but if they contain an unescaped separator character, would be treated as above. Since in the above suggestion, "##{Hello!};" would have the exact same effect as the already-supported "##(Hello!);", this may make more sense. The downsides would be that additional processing overhead would be added to parsing of the existing expression type, and of course, in the unlikely event that the chosen separator character is already used unescaped within literal expressions in existing scripts, these would need to be updated to include escape characters (a very minor point of backwards-compatibility which might not affect a single script).

A further consideration would be whether a mechanism should be implemented to allow entry weighting within these inline groups?

3) A "START" qualifier to extend current labelling capabilities (implemented!)

Currently, parsing of a Regnus script is started from a labelled line within the script, which is very powerful and flexible, but does lead to a certain amount of over-complexity, and often requires the creation of dummy groups in simple scripts. For example:

Code: Select all

GROUP StartGroup
ENTRY <Greeting>##Greetings;! My name is Elvis.

GROUP Greetings
ENTRY Hello
ENTRY Goodbye

This proposal would be for a new qualifier which would allow a start point to be specified without the creation of a containing group, allowing this script to be simplified to:

Code: Select all

START Greeting:##Greetings;! My name is Elvis.

GROUP Greetings
ENTRY Hello
ENTRY Goodbye

Or perhaps instead with the syntax:

Code: Select all

START <Example>Hello world!

4) The ability to use fractional values in entry weighting (implemented!)

This would allow the use of entry weighting to reduce rather than increase the chance of a particular entry being selected. For example:

Code: Select all

GROUP Cats
ENTRY tiger
ENTRY lion
RATIO 2:panther
RATIO 0.5:felis silvestris lybica

Here, the first two entries would each have a selection probability of 2/9, the third would have a probability of 4/9, and the final entry would have a probability of 1/9.

This would be useful in cases where an entry is to be added to an existing large group, for which it is desirable to assign a lower probability without the need to increase the ratios of all other entries in the group.

5) Functionality for simply generating random numerical values (implemented!)

It is currently possible to select random numbers from a group using Regnus, or even to assemble larger numbers from individual digits, but there is currently no simple functionality for generating random numbers within specific ranges or with specific probabilities, which would be useful (particularly when using a Regnus script to generate random encounter charts for RPGs etc.), and make for less complex script structures and therefore more readable code.

The proposal would be to add a new expression type whose contents would be parsed as a form of micro-syntax within Regnus, very possibly based on existing dice notation. So, for example:

Code: Select all

GROUP Examples
ENTRY A single four-sided die: ##{D4};
ENTRY Four six-sided dice: ##{4D6};
ENTRY Store the result of two ten-sided dice plus two in slot number five: #>(5):{2D10+2};

Note: the example above uses the same bracket type as one of the suggested forms for inline selection in suggestion #2, above, but this is just for simplicity of explanation; assuming both are implemented, they will of course need to use different syntax!

6) A way to specify entries whose content is entirely or partially escaped (not to be supported in Regnus 5.5)

Usually, the overstep AGC can be used to escape characters within an entry which would usually be understood (and therefore processed) by the parser as AGCs or other reserved characters. However, in scripts with a large amount of pre-formatted text containing a number of reserved characters, and little need for AGC functionality (particularly when using Regnus to generate (X)HTML, for example), this can be time-consuming and lead to convoluted code. Therefore, this suggestion is for a method of notating that a particular entry should not be parsed.

One possible solution would be to add a modifier to existing qualifiers to notate that they should not be parsed. For example:

Code: Select all

GROUP Examples
ENTRY This entry is parsed, thus *<strong>these instances of markup*</strong> need to be escaped with the ** character.
ENTRY! This entry is not parsed, thus <strong>this markup</strong> does not need to use the * character.
RATIO! 2:This entry has a 50% chance of being selected, and is also not parsed, thus <strong>this markup is <em>also</em></strong> unescaped.

One consideration is whether the effect of this format should be to cause the entry to be completely unparsed (including all references, etc.), or whether only AGCs, labels, or other specific reserved characters should be ignored? Perhaps multiple modifiers could be introduced to attain these results where appropriate?

Another consideration would be whether the overstep character could be used in unparsed entries for the reverse effect of allowing the parsing of a following reserved character?

Perhaps a new RTC could also be added which forces this effect on an entry which isn't otherwise marked as unparsed? Another possibility is that such an RTC could be used instead of specifying at the qualifier level.

[Ryan]

Update: points 1) and 2) updated, and point 3) added.

Ryan

[thunderclap]

First a quick suggestion. The new ways to capitalize are very good but one thing's still missing. A title case option to allow for grammatically correct capitalization of books and such.

Something else I've wanted for a while was a way to underweight an entry. Occasionally I come across a situation where I want just a few entries in a group to be much more rare than all the other entries in that group. The only way to actually do that is to set the ratio on all the other entries high. For an example

Code: Select all

GROUP Tools
ENTRY jackhammer
ENTRY hammer
ENTRY drill
ENTRY screwdriver
ENTRY sledgehammer
[b]ENTRY wrench
ENTRY socket wrench[/b]
ENTRY chisel
[b]ENTRY saw
ENTRY circular saw[/b]
ENTRY pickaxe
ENTRY razor
ENTRY scissors
ENTRY scythe
ENTRY wire cutter
ENTRY ice pick

In this particular case socket wrench is kind of redundant because there's also wrench, but at the same time it does add a little something different too. It would be nice to be able to have just socket wrench be more rare while keeping the ratio on everything else the same.

One solution would be to make a new group with just wrench and socket wrench (and wrench ratio set high), but that's a bit overkill and much more disorganized. Plus there's a similar situation with saw/circular saw, as well as hammer/sledge hammer, which means that would have to be done three times...

On the other hand upping the ratio on everything but a few entries is time consuming in comparison, and kind of a messy solution if you later have to do more fine-tuning of the ratios.

My suggestion for how to underweight would simply be to allow negative numbers for ratio. I'm not sure if that's ideal or not, but I'm going to go out on a limb and guess that decimals aren't something you'd want to add


Onto the points in the first post (I'll have to get to 1 some other time)

2) Being able to do things like
##{##SecondGroup¦##ThirdGroup¦##FourthGroup}; bananas.
and
##{##SecondGroup ##ThirdGroup¦##SecondGroup ##FourthGroup}; bananas.
would be very useful. Not having to make tiny one-time use groups anymore that get lost somewhere in the middle of a script would be so nice

3) Minor, but it does simplify things for simple scripts. Interestingly, 2) combined with 3) could potentially allow for scripts which had no groups at all, if they were simple enough.

Actually, an alternative might be to just ditch the old syntax of
GROUP GroupNameThat'sNeverActuallyUsed
ENTRY <label>stuff
and do something like
START
ENTRY label1:stuff
ENTRY label2:stuff
or
START
LABEL label1:stuff


Btw, is there a way to receive emails when threads you're watching are replied to?

[Ryan]

First a quick suggestion. The new ways to capitalize are very good but one thing's still missing. A title case option to allow for grammatically correct capitalization of books and such.

Actually, namecase is supported:

Code: Select all

ENTRY I just read ^(dirk gently's holistic detective agency).
BLANK Returns "I just read Dirk Gently's Holistic Detective Agency."

Something else I've wanted for a while was a way to underweight an entry. Occasionally I come across a situation where I want just a few entries in a group to be much more rare than all the other entries in that group. The only way to actually do that is to set the ratio on all the other entries high.

That's an interesting proposition. Though you might be surprised, I actually quite like the idea of allowing fractional ratios, since it's elegant and requires no syntactical changes to the existing system, so I'll consider this. Actually though, in your given example, I would probably use separate groups, as the different types of hammer could subcategorize quite nicely. In fact, it could be a good use of the suggested inline-selection expression, with some kind of ratio functionality added...

Btw, is there a way to receive emails when threads you're watching are replied to?

Yes - in the "Options" box at the bottom of the reply screen, just check the "Notify me when a reply is posted" box.

Ryan

[thunderclap]

That's strange. I had that set but never received emails for the march or april posts in this thread despite being subscribed to it (which is why I didn't know 5.4 was out until yesterday).

I did receive an email because of your response today though so it seems to be working right again.

Actually, namecase is supported:

Code: Select all

ENTRY I just read ^(dirk gently's holistic detective agency).
BLANK Returns "I just read Dirk Gently's Holistic Detective Agency."

Actually, I meant things like ^(of mice and men), where "and" shouldn't be capitalized in a title, and "of" normally wouldn't be either, but in this case because it's the first word of the title "of" should be as well. I don't know if it would be overkill to put that in Regnus but here's the most concise reference I could find.

Or for a quick list this page mentions "Capitalize all words in titles of publications and documents, except a, an, the, at, by, for, in, of, on, to, up, and, as, but, it, or, and nor." "From" would normally be lower too.

The first two advanced rules on that page would also be pretty easy to implement, if this were put in. The third rule not so much.

[thunderclap]

It seemed like this would be better as a separate post.

This part about Draft stood out:

Another important consideration regarding this proposed functionality is that it would almost certainly add additional loading time to scripts which don't even use drafting, and would add a level of complexity to the language which might be undesirable given that it adds a minor convenience for some authors, but no new functionality.

To get around extra overhead in scripts that don't use drafting, maybe scripts that do use drafting would have to specify that they use drafting at the start of the file. If that isn't specified in the file the user loads, Regnus could behave as it does now, as if drafting didn't exist.

There also might be no new functionality from a technical point, but if drafting were added it could potentially allow for a more communal/collaborative way of making scripts. It could also make things easier for users who might want to make changes to pre-existing scripts. Let's say Joe Shmoe likes the Random Nonsense script, but he'd also like Star Wars characters to be added to the famous people group. Right now he could add a bunch of Star Wars characters to the Nonsense script, save, and that's the end of it.

Except when Random Nonsense is updated on the site (let's say because of a Regnus syntax change), if he wants to keep those changes he needs to find what he all changed in the old version and paste it into the new version. Also, no one is befitting from any of these changes but him. I'll get back to that in a bit.

With drafting he'd be able to do it another way. He could put all the Star Wars characters into a separate script, add one line to Random Nonsense drafting in that script, and (assuming drafting would allow pre-existing Groups to be added to by drafted scripts) the most he would have to do again in the future is re-add that one line any time Random Nonsense is updated.

But he could also upload that Star Wars script if he wanted to, allowing anyone to easily have Star Wars characters show up in Random Nonsense too, just by adding in one line to draft the script. Let's say other people do the same with Star Trek, LOTR, Dr Who, etc. Now anyone can easily customize Nonsense to use them too. And because those lists exist and are easy to add, anyone making new scripts can use them too...

Now let's look at Billy. Billy's fussy and doesn't want to add a new line every time some puts up a new script he likes. He'd rather just draft in a folder where he puts all those scripts, and anything he puts in that folder will get loaded.

Billy's a really fussy boy though, and he just doesn't want to have to move files around to choose what gets used. (Maybe there could be checkboxes or something at runtime for people to select what drafted scripts they want/don't want used?)

I'm aware there'd be (possibly major) complications with allowing thing to be done that way. That kind of dynamic flexibility of scripts is probably not something you ever intended for Regnus either. I'm not so sure though that "minor convenience for some authors, but no new functionality" is necessarily true.

[Ryan]

Actually, I meant things like ^(of mice and men), where "and" shouldn't be capitalized in a title, and "of" normally wouldn't be either, but in this case because it's the first word of the title "of" should be as well.

Ah, this is very much a matter of style-guide, and really isn't any kind of "rule" (personally, I tend to capitalise all words in titles unless purely for aesthetic effect in design presentation). It would be possible to add a further capitalisation option which allowed arbitrary words to be excluded, but could potentially be over-complex as it would need to allow for different authors' choices of style regarding which words should be excluded...

To get around extra overhead in scripts that don't use drafting, maybe scripts that do use drafting would have to specify that they use drafting at the start of the file.

That's not a solution I like - it feels a little like a hack. I think the more elegant approach would be to try to find a way to implement the functionality efficiently in the parser, rather than add additional requirements to script authors.

There also might be no new functionality from a technical point, but if drafting were added it could potentially allow for a more communal/collaborative way of making scripts.

That's certainly true - it would allow script snippets to be very easily sharable, and I think that's a cool idea; I'm just not 100% sure that it provides a great deal more functionality in that instance than adding a weighted entry to the existing group containing a reference to the group in the external script, given the amount of extra complexity it would add to the parser in order to achieve it.

Now let's look at Billy. Billy's fussy and doesn't want to add a new line every time some puts up a new script he likes. He'd rather just draft in a folder where he puts all those scripts, and anything he puts in that folder will get loaded.

Billy's a really fussy boy though, and he just doesn't want to have to move files around to choose what gets used. (Maybe there could be checkboxes or something at runtime for people to select what drafted scripts they want/don't want used?)

I certainly wouldn't want to add functionality which required the parser to read directory listings into a script. Besides the complexity, I think that would definitely be beyond the remit of the new qualifier, and would lead to all sorts of additional issues, not to mention cross-platform compatibility considerations. And adding user-input to choose which draft qualifiers to use would definitely be overkill, I think.

Ryan

[Ryan]

Update: point 4) added, and further consideration added to point 2).

Ryan

[thunderclap]

I believe you meant point 2 got further consideration, not 3

As the whole point of inline groups is to avoid having to make tiny one-time-use groups, it would make sense to be able to weight there too. Otherwise any time weighting would have been needed someone would pretty much have to make a separate group anyway. A quick example, if someone wanted to choose between weekday or weekend, but wanted things weighted 2:weekend and 5:weekday.

[Ryan]

I believe you meant point 2 got further consideration, not 3

Good spot - corrected.

As the whole point of inline groups is to avoid having to make tiny one-time-use groups, it would make sense to be able to weight there too. Otherwise any time weighting would have been needed someone would pretty much have to make a separate group anyway. A quick example, if someone wanted to choose between weekday or weekend, but wanted things weighted 2:weekend and 5:weekday.

Yes, that's certainly my thinking. But another possible school of thought here would be that the inline selection should remain very simple in its functionality, with more complex selections being made using the traditional method. Personally, I think some basic level of weighting is a very good idea to include, but adding functionality to the inline method could be considered a slippery slope towards encouraging convoluted and messy scripting.

The other thing to consider, of course, is what syntax to use for inline weighting, in order to avoid ambiguities and conflicts within the script.

Ryan