where to place a Config-Diff algorigthm

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

where to place a Config-Diff algorigthm

Mark Struberg-3
Hi folks!

For the new DS-1.9 feature to 'push' config changes we need an algorithm to detect whether an old and a new config differs.

The signature would be something like:

/**
 *
 * A Set of all the attributes which differ between the old and new config Map. An empty Set if there is no difference.
 */
public Set<String> diffConfig(Map<String, String> oldValues, Map<String, String> newValues)

This is intended for e.g. background threads which read from a database once per second and compare the old values with the new ones.
If there was any difference then the set of attributes get reported back to the Config (which in turn clears the caches, etc).


Now where to put this method?

My candidate would be org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
I do not want to put it into the Config interface itself because it is not a user contract thingy.
And I also do not want to put it into ConfigResolver becasue I'd like to have the impl only available internally and not bloat the ConfigResolver any further.

Wdyt?

LieGrue,
strub
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Jean-Louis MONTEIRO
Few thoughts ...

I did not follow really deltaspike to be honest, but I do follow MP-Config
for instance.

First it's all about config, so maybe having a method name "diff" is
enough. No need for "diffConfig".

Some values may have been updated, some might have been deleted and some
other added.
Returning a set might not be enough, isn't it?

Jean-Louis






Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
écrit :

> Hi folks!
>
> For the new DS-1.9 feature to 'push' config changes we need an algorithm
> to detect whether an old and a new config differs.
>
> The signature would be something like:
>
> /**
>  *
>  * A Set of all the attributes which differ between the old and new config
> Map. An empty Set if there is no difference.
>  */
> public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
> String> newValues)
>
> This is intended for e.g. background threads which read from a database
> once per second and compare the old values with the new ones.
> If there was any difference then the set of attributes get reported back
> to the Config (which in turn clears the caches, etc).
>
>
> Now where to put this method?
>
> My candidate would be
> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
> I do not want to put it into the Config interface itself because it is not
> a user contract thingy.
> And I also do not want to put it into ConfigResolver becasue I'd like to
> have the impl only available internally and not bloat the ConfigResolver
> any further.
>
> Wdyt?
>
> LieGrue,
> strub
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Romain Manni-Bucau
Hi Mark,

Personally I see it as an internal of a source which fakes to be a map but
is actually key based (= not able to load all values up to date at once
like a database)
so i'm tempted to say we should do like for EE projects and create a
config-source module with all the utilities for users in a portable way
(today a user requires to use impl which is not as clean as we are normally
since it mixes extensible classes by users and internals).

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 5 juin 2018 à 09:00, Jean-Louis MONTEIRO <[hidden email]> a
écrit :

> Few thoughts ...
>
> I did not follow really deltaspike to be honest, but I do follow MP-Config
> for instance.
>
> First it's all about config, so maybe having a method name "diff" is
> enough. No need for "diffConfig".
>
> Some values may have been updated, some might have been deleted and some
> other added.
> Returning a set might not be enough, isn't it?
>
> Jean-Louis
>
>
>
>
>
>
> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
> écrit :
>
> > Hi folks!
> >
> > For the new DS-1.9 feature to 'push' config changes we need an algorithm
> > to detect whether an old and a new config differs.
> >
> > The signature would be something like:
> >
> > /**
> >  *
> >  * A Set of all the attributes which differ between the old and new
> config
> > Map. An empty Set if there is no difference.
> >  */
> > public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
> > String> newValues)
> >
> > This is intended for e.g. background threads which read from a database
> > once per second and compare the old values with the new ones.
> > If there was any difference then the set of attributes get reported back
> > to the Config (which in turn clears the caches, etc).
> >
> >
> > Now where to put this method?
> >
> > My candidate would be
> > org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
> > I do not want to put it into the Config interface itself because it is
> not
> > a user contract thingy.
> > And I also do not want to put it into ConfigResolver becasue I'd like to
> > have the impl only available internally and not bloat the ConfigResolver
> > any further.
> >
> > Wdyt?
> >
> > LieGrue,
> > strub
>
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Mark Struberg-3
In reply to this post by Jean-Louis MONTEIRO
Hi JL!

Thanks for the feedback!
And yes, this will also be important for ConfigJSR (and later mp-config as well).


> First it's all about config, so maybe having a method name "diff" is
> enough. No need for "diffConfig".

Well, diff alone is imo a bit too short as it doesn't have any context about what to diff.
Thus I'd prefer diffConfig.


> Some values may have been updated, some might have been deleted and some
> other added.
> Returning a set might not be enough, isn't it?

We really only need the property names which got changed:
* new ones
* deleted ones
* the ones with different values

So a Set<String> is fine.

LieGrue,
strub


> Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <[hidden email]>:
>
> Few thoughts ...
>
> I did not follow really deltaspike to be honest, but I do follow MP-Config
> for instance.
>
> First it's all about config, so maybe having a method name "diff" is
> enough. No need for "diffConfig".
>
> Some values may have been updated, some might have been deleted and some
> other added.
> Returning a set might not be enough, isn't it?
>
> Jean-Louis
>
>
>
>
>
>
> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
> écrit :
>
>> Hi folks!
>>
>> For the new DS-1.9 feature to 'push' config changes we need an algorithm
>> to detect whether an old and a new config differs.
>>
>> The signature would be something like:
>>
>> /**
>> *
>> * A Set of all the attributes which differ between the old and new config
>> Map. An empty Set if there is no difference.
>> */
>> public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
>> String> newValues)
>>
>> This is intended for e.g. background threads which read from a database
>> once per second and compare the old values with the new ones.
>> If there was any difference then the set of attributes get reported back
>> to the Config (which in turn clears the caches, etc).
>>
>>
>> Now where to put this method?
>>
>> My candidate would be
>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
>> I do not want to put it into the Config interface itself because it is not
>> a user contract thingy.
>> And I also do not want to put it into ConfigResolver becasue I'd like to
>> have the impl only available internally and not bloat the ConfigResolver
>> any further.
>>
>> Wdyt?
>>
>> LieGrue,
>> strub

Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Mark Struberg-3
In reply to this post by Romain Manni-Bucau
I thought about this as well, but would love to hide all those things behind an interface.

I thought about introducing a

/**
 * @param reloadEach if null, then only read once at startup, if Long is given then start a background thread to scan for changes every reloadEach ms.
 */
ConfigSource ConfigProvider#newConfigSource(URL fromUrl, Long reloadEach);

Wdyt?

LieGrue,
strub



> Am 05.06.2018 um 09:03 schrieb Romain Manni-Bucau <[hidden email]>:
>
> Hi Mark,
>
> Personally I see it as an internal of a source which fakes to be a map but
> is actually key based (= not able to load all values up to date at once
> like a database)
> so i'm tempted to say we should do like for EE projects and create a
> config-source module with all the utilities for users in a portable way
> (today a user requires to use impl which is not as clean as we are normally
> since it mixes extensible classes by users and internals).
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le mar. 5 juin 2018 à 09:00, Jean-Louis MONTEIRO <[hidden email]> a
> écrit :
>
>> Few thoughts ...
>>
>> I did not follow really deltaspike to be honest, but I do follow MP-Config
>> for instance.
>>
>> First it's all about config, so maybe having a method name "diff" is
>> enough. No need for "diffConfig".
>>
>> Some values may have been updated, some might have been deleted and some
>> other added.
>> Returning a set might not be enough, isn't it?
>>
>> Jean-Louis
>>
>>
>>
>>
>>
>>
>> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
>> écrit :
>>
>>> Hi folks!
>>>
>>> For the new DS-1.9 feature to 'push' config changes we need an algorithm
>>> to detect whether an old and a new config differs.
>>>
>>> The signature would be something like:
>>>
>>> /**
>>> *
>>> * A Set of all the attributes which differ between the old and new
>> config
>>> Map. An empty Set if there is no difference.
>>> */
>>> public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
>>> String> newValues)
>>>
>>> This is intended for e.g. background threads which read from a database
>>> once per second and compare the old values with the new ones.
>>> If there was any difference then the set of attributes get reported back
>>> to the Config (which in turn clears the caches, etc).
>>>
>>>
>>> Now where to put this method?
>>>
>>> My candidate would be
>>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
>>> I do not want to put it into the Config interface itself because it is
>> not
>>> a user contract thingy.
>>> And I also do not want to put it into ConfigResolver becasue I'd like to
>>> have the impl only available internally and not bloat the ConfigResolver
>>> any further.
>>>
>>> Wdyt?
>>>
>>> LieGrue,
>>> strub
>>

Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Jean-Louis MONTEIRO
In reply to this post by Mark Struberg-3
>> Some values may have been updated, some might have been deleted and some
>> other added.
>> Returning a set might not be enough, isn't it?

>We really only need the property names which got changed:
>* new ones
>* deleted ones
>* the ones with different values
>
>So a Set<String> is fine.

The thing is that you will get the key of everything that changed which
requires to invalidate the cache for all entries (lock?).
Or you would need to iterate all the keys, compare again old/new and delete
entries from cache, do nothing for add (? they will be lazily added during
lookup maybe) and invalidate the updated values.

That seems to be heavy to handle for nothing has during compare we have all
information already.



Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]> a
écrit :

> Hi JL!
>
> Thanks for the feedback!
> And yes, this will also be important for ConfigJSR (and later mp-config as
> well).
>
>
> > First it's all about config, so maybe having a method name "diff" is
> > enough. No need for "diffConfig".
>
> Well, diff alone is imo a bit too short as it doesn't have any context
> about what to diff.
> Thus I'd prefer diffConfig.
>
>
> > Some values may have been updated, some might have been deleted and some
> > other added.
> > Returning a set might not be enough, isn't it?
>
> We really only need the property names which got changed:
> * new ones
> * deleted ones
> * the ones with different values
>
> So a Set<String> is fine.
>
> LieGrue,
> strub
>
>
> > Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <[hidden email]>:
> >
> > Few thoughts ...
> >
> > I did not follow really deltaspike to be honest, but I do follow
> MP-Config
> > for instance.
> >
> > First it's all about config, so maybe having a method name "diff" is
> > enough. No need for "diffConfig".
> >
> > Some values may have been updated, some might have been deleted and some
> > other added.
> > Returning a set might not be enough, isn't it?
> >
> > Jean-Louis
> >
> >
> >
> >
> >
> >
> > Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
> > écrit :
> >
> >> Hi folks!
> >>
> >> For the new DS-1.9 feature to 'push' config changes we need an algorithm
> >> to detect whether an old and a new config differs.
> >>
> >> The signature would be something like:
> >>
> >> /**
> >> *
> >> * A Set of all the attributes which differ between the old and new
> config
> >> Map. An empty Set if there is no difference.
> >> */
> >> public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
> >> String> newValues)
> >>
> >> This is intended for e.g. background threads which read from a database
> >> once per second and compare the old values with the new ones.
> >> If there was any difference then the set of attributes get reported back
> >> to the Config (which in turn clears the caches, etc).
> >>
> >>
> >> Now where to put this method?
> >>
> >> My candidate would be
> >> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
> >> I do not want to put it into the Config interface itself because it is
> not
> >> a user contract thingy.
> >> And I also do not want to put it into ConfigResolver becasue I'd like to
> >> have the impl only available internally and not bloat the ConfigResolver
> >> any further.
> >>
> >> Wdyt?
> >>
> >> LieGrue,
> >> strub
>
>
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Romain Manni-Bucau
Hmm, maybe too early so don't hesitate to shout if studid: why not having a
config manager sources can register in it and if this manager has >= 1
source then the thread is active otherwise it is destroyed
(running.set(false))
The source would be able to pass its refresh logic (as a function more or
less) the thread will execute in a safe manner (if fail - log but dont quit
the loop) and it returns a boolean to ask to propagate changes upstream or
not.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mar. 5 juin 2018 à 09:23, Jean-Louis MONTEIRO <[hidden email]> a
écrit :

> >> Some values may have been updated, some might have been deleted and some
> >> other added.
> >> Returning a set might not be enough, isn't it?
>
> >We really only need the property names which got changed:
> >* new ones
> >* deleted ones
> >* the ones with different values
> >
> >So a Set<String> is fine.
>
> The thing is that you will get the key of everything that changed which
> requires to invalidate the cache for all entries (lock?).
> Or you would need to iterate all the keys, compare again old/new and delete
> entries from cache, do nothing for add (? they will be lazily added during
> lookup maybe) and invalidate the updated values.
>
> That seems to be heavy to handle for nothing has during compare we have all
> information already.
>
>
>
> Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]> a
> écrit :
>
> > Hi JL!
> >
> > Thanks for the feedback!
> > And yes, this will also be important for ConfigJSR (and later mp-config
> as
> > well).
> >
> >
> > > First it's all about config, so maybe having a method name "diff" is
> > > enough. No need for "diffConfig".
> >
> > Well, diff alone is imo a bit too short as it doesn't have any context
> > about what to diff.
> > Thus I'd prefer diffConfig.
> >
> >
> > > Some values may have been updated, some might have been deleted and
> some
> > > other added.
> > > Returning a set might not be enough, isn't it?
> >
> > We really only need the property names which got changed:
> > * new ones
> > * deleted ones
> > * the ones with different values
> >
> > So a Set<String> is fine.
> >
> > LieGrue,
> > strub
> >
> >
> > > Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <[hidden email]
> >:
> > >
> > > Few thoughts ...
> > >
> > > I did not follow really deltaspike to be honest, but I do follow
> > MP-Config
> > > for instance.
> > >
> > > First it's all about config, so maybe having a method name "diff" is
> > > enough. No need for "diffConfig".
> > >
> > > Some values may have been updated, some might have been deleted and
> some
> > > other added.
> > > Returning a set might not be enough, isn't it?
> > >
> > > Jean-Louis
> > >
> > >
> > >
> > >
> > >
> > >
> > > Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]>
> a
> > > écrit :
> > >
> > >> Hi folks!
> > >>
> > >> For the new DS-1.9 feature to 'push' config changes we need an
> algorithm
> > >> to detect whether an old and a new config differs.
> > >>
> > >> The signature would be something like:
> > >>
> > >> /**
> > >> *
> > >> * A Set of all the attributes which differ between the old and new
> > config
> > >> Map. An empty Set if there is no difference.
> > >> */
> > >> public Set<String> diffConfig(Map<String, String> oldValues,
> Map<String,
> > >> String> newValues)
> > >>
> > >> This is intended for e.g. background threads which read from a
> database
> > >> once per second and compare the old values with the new ones.
> > >> If there was any difference then the set of attributes get reported
> back
> > >> to the Config (which in turn clears the caches, etc).
> > >>
> > >>
> > >> Now where to put this method?
> > >>
> > >> My candidate would be
> > >> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
> > >> I do not want to put it into the Config interface itself because it is
> > not
> > >> a user contract thingy.
> > >> And I also do not want to put it into ConfigResolver becasue I'd like
> to
> > >> have the impl only available internally and not bloat the
> ConfigResolver
> > >> any further.
> > >>
> > >> Wdyt?
> > >>
> > >> LieGrue,
> > >> strub
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Mark Struberg-3
In reply to this post by Jean-Louis MONTEIRO
Currently it only sets the lastChanged timestamp on ANY change.
https://github.com/apache/deltaspike/blob/master/deltaspike/core/impl/src/main/java/org/apache/deltaspike/core/impl/config/ConfigImpl.java#L239

This gets evaluated during the cache access:
https://github.com/apache/deltaspike/blob/master/deltaspike/core/impl/src/main/java/org/apache/deltaspike/core/impl/config/TypedResolverImpl.java#L245

and for snapshots:
https://github.com/apache/deltaspike/blob/master/deltaspike/core/impl/src/main/java/org/apache/deltaspike/core/impl/config/ConfigImpl.java#L133

LieGrue,
strub


> Am 05.06.2018 um 09:22 schrieb Jean-Louis MONTEIRO <[hidden email]>:
>
>>> Some values may have been updated, some might have been deleted and some
>>> other added.
>>> Returning a set might not be enough, isn't it?
>
>> We really only need the property names which got changed:
>> * new ones
>> * deleted ones
>> * the ones with different values
>>
>> So a Set<String> is fine.
>
> The thing is that you will get the key of everything that changed which
> requires to invalidate the cache for all entries (lock?).
> Or you would need to iterate all the keys, compare again old/new and delete
> entries from cache, do nothing for add (? they will be lazily added during
> lookup maybe) and invalidate the updated values.
>
> That seems to be heavy to handle for nothing has during compare we have all
> information already.
>
>
>
> Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]> a
> écrit :
>
>> Hi JL!
>>
>> Thanks for the feedback!
>> And yes, this will also be important for ConfigJSR (and later mp-config as
>> well).
>>
>>
>>> First it's all about config, so maybe having a method name "diff" is
>>> enough. No need for "diffConfig".
>>
>> Well, diff alone is imo a bit too short as it doesn't have any context
>> about what to diff.
>> Thus I'd prefer diffConfig.
>>
>>
>>> Some values may have been updated, some might have been deleted and some
>>> other added.
>>> Returning a set might not be enough, isn't it?
>>
>> We really only need the property names which got changed:
>> * new ones
>> * deleted ones
>> * the ones with different values
>>
>> So a Set<String> is fine.
>>
>> LieGrue,
>> strub
>>
>>
>>> Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <[hidden email]>:
>>>
>>> Few thoughts ...
>>>
>>> I did not follow really deltaspike to be honest, but I do follow
>> MP-Config
>>> for instance.
>>>
>>> First it's all about config, so maybe having a method name "diff" is
>>> enough. No need for "diffConfig".
>>>
>>> Some values may have been updated, some might have been deleted and some
>>> other added.
>>> Returning a set might not be enough, isn't it?
>>>
>>> Jean-Louis
>>>
>>>
>>>
>>>
>>>
>>>
>>> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]> a
>>> écrit :
>>>
>>>> Hi folks!
>>>>
>>>> For the new DS-1.9 feature to 'push' config changes we need an algorithm
>>>> to detect whether an old and a new config differs.
>>>>
>>>> The signature would be something like:
>>>>
>>>> /**
>>>> *
>>>> * A Set of all the attributes which differ between the old and new
>> config
>>>> Map. An empty Set if there is no difference.
>>>> */
>>>> public Set<String> diffConfig(Map<String, String> oldValues, Map<String,
>>>> String> newValues)
>>>>
>>>> This is intended for e.g. background threads which read from a database
>>>> once per second and compare the old values with the new ones.
>>>> If there was any difference then the set of attributes get reported back
>>>> to the Config (which in turn clears the caches, etc).
>>>>
>>>>
>>>> Now where to put this method?
>>>>
>>>> My candidate would be
>>>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
>>>> I do not want to put it into the Config interface itself because it is
>> not
>>>> a user contract thingy.
>>>> And I also do not want to put it into ConfigResolver becasue I'd like to
>>>> have the impl only available internally and not bloat the ConfigResolver
>>>> any further.
>>>>
>>>> Wdyt?
>>>>
>>>> LieGrue,
>>>> strub
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Mark Struberg-3
In reply to this post by Romain Manni-Bucau
Interesting solution, but I fear there are 2 problems with this:
a.) It won't be backward compatible with existing ConfigSources out in the wild
b.) It's likely an overkill. That solution would make sense if you have a tons of changes, but not if there is barely something which moves.

LieGrue,
strub

> Am 05.06.2018 um 09:34 schrieb Romain Manni-Bucau <[hidden email]>:
>
> Hmm, maybe too early so don't hesitate to shout if studid: why not having a
> config manager sources can register in it and if this manager has >= 1
> source then the thread is active otherwise it is destroyed
> (running.set(false))
> The source would be able to pass its refresh logic (as a function more or
> less) the thread will execute in a safe manner (if fail - log but dont quit
> the loop) and it returns a boolean to ask to propagate changes upstream or
> not.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le mar. 5 juin 2018 à 09:23, Jean-Louis MONTEIRO <[hidden email]> a
> écrit :
>
>>>> Some values may have been updated, some might have been deleted and some
>>>> other added.
>>>> Returning a set might not be enough, isn't it?
>>
>>> We really only need the property names which got changed:
>>> * new ones
>>> * deleted ones
>>> * the ones with different values
>>>
>>> So a Set<String> is fine.
>>
>> The thing is that you will get the key of everything that changed which
>> requires to invalidate the cache for all entries (lock?).
>> Or you would need to iterate all the keys, compare again old/new and delete
>> entries from cache, do nothing for add (? they will be lazily added during
>> lookup maybe) and invalidate the updated values.
>>
>> That seems to be heavy to handle for nothing has during compare we have all
>> information already.
>>
>>
>>
>> Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]> a
>> écrit :
>>
>>> Hi JL!
>>>
>>> Thanks for the feedback!
>>> And yes, this will also be important for ConfigJSR (and later mp-config
>> as
>>> well).
>>>
>>>
>>>> First it's all about config, so maybe having a method name "diff" is
>>>> enough. No need for "diffConfig".
>>>
>>> Well, diff alone is imo a bit too short as it doesn't have any context
>>> about what to diff.
>>> Thus I'd prefer diffConfig.
>>>
>>>
>>>> Some values may have been updated, some might have been deleted and
>> some
>>>> other added.
>>>> Returning a set might not be enough, isn't it?
>>>
>>> We really only need the property names which got changed:
>>> * new ones
>>> * deleted ones
>>> * the ones with different values
>>>
>>> So a Set<String> is fine.
>>>
>>> LieGrue,
>>> strub
>>>
>>>
>>>> Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <[hidden email]
>>> :
>>>>
>>>> Few thoughts ...
>>>>
>>>> I did not follow really deltaspike to be honest, but I do follow
>>> MP-Config
>>>> for instance.
>>>>
>>>> First it's all about config, so maybe having a method name "diff" is
>>>> enough. No need for "diffConfig".
>>>>
>>>> Some values may have been updated, some might have been deleted and
>> some
>>>> other added.
>>>> Returning a set might not be enough, isn't it?
>>>>
>>>> Jean-Louis
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]>
>> a
>>>> écrit :
>>>>
>>>>> Hi folks!
>>>>>
>>>>> For the new DS-1.9 feature to 'push' config changes we need an
>> algorithm
>>>>> to detect whether an old and a new config differs.
>>>>>
>>>>> The signature would be something like:
>>>>>
>>>>> /**
>>>>> *
>>>>> * A Set of all the attributes which differ between the old and new
>>> config
>>>>> Map. An empty Set if there is no difference.
>>>>> */
>>>>> public Set<String> diffConfig(Map<String, String> oldValues,
>> Map<String,
>>>>> String> newValues)
>>>>>
>>>>> This is intended for e.g. background threads which read from a
>> database
>>>>> once per second and compare the old values with the new ones.
>>>>> If there was any difference then the set of attributes get reported
>> back
>>>>> to the Config (which in turn clears the caches, etc).
>>>>>
>>>>>
>>>>> Now where to put this method?
>>>>>
>>>>> My candidate would be
>>>>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
>>>>> I do not want to put it into the Config interface itself because it is
>>> not
>>>>> a user contract thingy.
>>>>> And I also do not want to put it into ConfigResolver becasue I'd like
>> to
>>>>> have the impl only available internally and not bloat the
>> ConfigResolver
>>>>> any further.
>>>>>
>>>>> Wdyt?
>>>>>
>>>>> LieGrue,
>>>>> strub
>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Romain Manni-Bucau
Le mar. 5 juin 2018 à 10:25, Mark Struberg <[hidden email]> a
écrit :

> Interesting solution, but I fear there are 2 problems with this:
> a.) It won't be backward compatible with existing ConfigSources out in the
> wild
>

Well we don't support it and the lasttimestamp solution has the same issue
so kind of status quo ;)


> b.) It's likely an overkill. That solution would make sense if you have a
> tons of changes, but not if there is barely something which moves.
>

Not really. it is more about: is the source pushing its changes or the
config polling them. I think the push solution is saner and in this case
doesn't require the polling and lasttimestamp at all, no?


>
> LieGrue,
> strub
>
> > Am 05.06.2018 um 09:34 schrieb Romain Manni-Bucau <[hidden email]
> >:
> >
> > Hmm, maybe too early so don't hesitate to shout if studid: why not
> having a
> > config manager sources can register in it and if this manager has >= 1
> > source then the thread is active otherwise it is destroyed
> > (running.set(false))
> > The source would be able to pass its refresh logic (as a function more or
> > less) the thread will execute in a safe manner (if fail - log but dont
> quit
> > the loop) and it returns a boolean to ask to propagate changes upstream
> or
> > not.
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le mar. 5 juin 2018 à 09:23, Jean-Louis MONTEIRO <[hidden email]> a
> > écrit :
> >
> >>>> Some values may have been updated, some might have been deleted and
> some
> >>>> other added.
> >>>> Returning a set might not be enough, isn't it?
> >>
> >>> We really only need the property names which got changed:
> >>> * new ones
> >>> * deleted ones
> >>> * the ones with different values
> >>>
> >>> So a Set<String> is fine.
> >>
> >> The thing is that you will get the key of everything that changed which
> >> requires to invalidate the cache for all entries (lock?).
> >> Or you would need to iterate all the keys, compare again old/new and
> delete
> >> entries from cache, do nothing for add (? they will be lazily added
> during
> >> lookup maybe) and invalidate the updated values.
> >>
> >> That seems to be heavy to handle for nothing has during compare we have
> all
> >> information already.
> >>
> >>
> >>
> >> Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]>
> a
> >> écrit :
> >>
> >>> Hi JL!
> >>>
> >>> Thanks for the feedback!
> >>> And yes, this will also be important for ConfigJSR (and later mp-config
> >> as
> >>> well).
> >>>
> >>>
> >>>> First it's all about config, so maybe having a method name "diff" is
> >>>> enough. No need for "diffConfig".
> >>>
> >>> Well, diff alone is imo a bit too short as it doesn't have any context
> >>> about what to diff.
> >>> Thus I'd prefer diffConfig.
> >>>
> >>>
> >>>> Some values may have been updated, some might have been deleted and
> >> some
> >>>> other added.
> >>>> Returning a set might not be enough, isn't it?
> >>>
> >>> We really only need the property names which got changed:
> >>> * new ones
> >>> * deleted ones
> >>> * the ones with different values
> >>>
> >>> So a Set<String> is fine.
> >>>
> >>> LieGrue,
> >>> strub
> >>>
> >>>
> >>>> Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <
> [hidden email]
> >>> :
> >>>>
> >>>> Few thoughts ...
> >>>>
> >>>> I did not follow really deltaspike to be honest, but I do follow
> >>> MP-Config
> >>>> for instance.
> >>>>
> >>>> First it's all about config, so maybe having a method name "diff" is
> >>>> enough. No need for "diffConfig".
> >>>>
> >>>> Some values may have been updated, some might have been deleted and
> >> some
> >>>> other added.
> >>>> Returning a set might not be enough, isn't it?
> >>>>
> >>>> Jean-Louis
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]
> >
> >> a
> >>>> écrit :
> >>>>
> >>>>> Hi folks!
> >>>>>
> >>>>> For the new DS-1.9 feature to 'push' config changes we need an
> >> algorithm
> >>>>> to detect whether an old and a new config differs.
> >>>>>
> >>>>> The signature would be something like:
> >>>>>
> >>>>> /**
> >>>>> *
> >>>>> * A Set of all the attributes which differ between the old and new
> >>> config
> >>>>> Map. An empty Set if there is no difference.
> >>>>> */
> >>>>> public Set<String> diffConfig(Map<String, String> oldValues,
> >> Map<String,
> >>>>> String> newValues)
> >>>>>
> >>>>> This is intended for e.g. background threads which read from a
> >> database
> >>>>> once per second and compare the old values with the new ones.
> >>>>> If there was any difference then the set of attributes get reported
> >> back
> >>>>> to the Config (which in turn clears the caches, etc).
> >>>>>
> >>>>>
> >>>>> Now where to put this method?
> >>>>>
> >>>>> My candidate would be
> >>>>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
> >>>>> I do not want to put it into the Config interface itself because it
> is
> >>> not
> >>>>> a user contract thingy.
> >>>>> And I also do not want to put it into ConfigResolver becasue I'd like
> >> to
> >>>>> have the impl only available internally and not bloat the
> >> ConfigResolver
> >>>>> any further.
> >>>>>
> >>>>> Wdyt?
> >>>>>
> >>>>> LieGrue,
> >>>>> strub
> >>>
> >>>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: where to place a Config-Diff algorigthm

Mark Struberg-3
Oh, we DO push from the ConfigSource to Config in the current solution!
It's just that we _right now_ do only leverage this for eagerly invalidating our caches.

We could later also introduce active notifications based on the infos we gather.
But there are many things we need to clarify before doing so.

LieGrue,
strub



> Am 05.06.2018 um 10:33 schrieb Romain Manni-Bucau <[hidden email]>:
>
> Le mar. 5 juin 2018 à 10:25, Mark Struberg <[hidden email]> a
> écrit :
>
>> Interesting solution, but I fear there are 2 problems with this:
>> a.) It won't be backward compatible with existing ConfigSources out in the
>> wild
>>
>
> Well we don't support it and the lasttimestamp solution has the same issue
> so kind of status quo ;)
>
>
>> b.) It's likely an overkill. That solution would make sense if you have a
>> tons of changes, but not if there is barely something which moves.
>>
>
> Not really. it is more about: is the source pushing its changes or the
> config polling them. I think the push solution is saner and in this case
> doesn't require the polling and lasttimestamp at all, no?
>
>
>>
>> LieGrue,
>> strub
>>
>>> Am 05.06.2018 um 09:34 schrieb Romain Manni-Bucau <[hidden email]
>>> :
>>>
>>> Hmm, maybe too early so don't hesitate to shout if studid: why not
>> having a
>>> config manager sources can register in it and if this manager has >= 1
>>> source then the thread is active otherwise it is destroyed
>>> (running.set(false))
>>> The source would be able to pass its refresh logic (as a function more or
>>> less) the thread will execute in a safe manner (if fail - log but dont
>> quit
>>> the loop) and it returns a boolean to ask to propagate changes upstream
>> or
>>> not.
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github <
>> https://github.com/rmannibucau> |
>>> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
>>> <
>> https://www.packtpub.com/application-development/java-ee-8-high-performance
>>>
>>>
>>>
>>> Le mar. 5 juin 2018 à 09:23, Jean-Louis MONTEIRO <[hidden email]> a
>>> écrit :
>>>
>>>>>> Some values may have been updated, some might have been deleted and
>> some
>>>>>> other added.
>>>>>> Returning a set might not be enough, isn't it?
>>>>
>>>>> We really only need the property names which got changed:
>>>>> * new ones
>>>>> * deleted ones
>>>>> * the ones with different values
>>>>>
>>>>> So a Set<String> is fine.
>>>>
>>>> The thing is that you will get the key of everything that changed which
>>>> requires to invalidate the cache for all entries (lock?).
>>>> Or you would need to iterate all the keys, compare again old/new and
>> delete
>>>> entries from cache, do nothing for add (? they will be lazily added
>> during
>>>> lookup maybe) and invalidate the updated values.
>>>>
>>>> That seems to be heavy to handle for nothing has during compare we have
>> all
>>>> information already.
>>>>
>>>>
>>>>
>>>> Le mar. 5 juin 2018 à 09:10, Mark Struberg <[hidden email]>
>> a
>>>> écrit :
>>>>
>>>>> Hi JL!
>>>>>
>>>>> Thanks for the feedback!
>>>>> And yes, this will also be important for ConfigJSR (and later mp-config
>>>> as
>>>>> well).
>>>>>
>>>>>
>>>>>> First it's all about config, so maybe having a method name "diff" is
>>>>>> enough. No need for "diffConfig".
>>>>>
>>>>> Well, diff alone is imo a bit too short as it doesn't have any context
>>>>> about what to diff.
>>>>> Thus I'd prefer diffConfig.
>>>>>
>>>>>
>>>>>> Some values may have been updated, some might have been deleted and
>>>> some
>>>>>> other added.
>>>>>> Returning a set might not be enough, isn't it?
>>>>>
>>>>> We really only need the property names which got changed:
>>>>> * new ones
>>>>> * deleted ones
>>>>> * the ones with different values
>>>>>
>>>>> So a Set<String> is fine.
>>>>>
>>>>> LieGrue,
>>>>> strub
>>>>>
>>>>>
>>>>>> Am 05.06.2018 um 08:59 schrieb Jean-Louis MONTEIRO <
>> [hidden email]
>>>>> :
>>>>>>
>>>>>> Few thoughts ...
>>>>>>
>>>>>> I did not follow really deltaspike to be honest, but I do follow
>>>>> MP-Config
>>>>>> for instance.
>>>>>>
>>>>>> First it's all about config, so maybe having a method name "diff" is
>>>>>> enough. No need for "diffConfig".
>>>>>>
>>>>>> Some values may have been updated, some might have been deleted and
>>>> some
>>>>>> other added.
>>>>>> Returning a set might not be enough, isn't it?
>>>>>>
>>>>>> Jean-Louis
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Le mar. 5 juin 2018 à 08:55, Mark Struberg <[hidden email]
>>>
>>>> a
>>>>>> écrit :
>>>>>>
>>>>>>> Hi folks!
>>>>>>>
>>>>>>> For the new DS-1.9 feature to 'push' config changes we need an
>>>> algorithm
>>>>>>> to detect whether an old and a new config differs.
>>>>>>>
>>>>>>> The signature would be something like:
>>>>>>>
>>>>>>> /**
>>>>>>> *
>>>>>>> * A Set of all the attributes which differ between the old and new
>>>>> config
>>>>>>> Map. An empty Set if there is no difference.
>>>>>>> */
>>>>>>> public Set<String> diffConfig(Map<String, String> oldValues,
>>>> Map<String,
>>>>>>> String> newValues)
>>>>>>>
>>>>>>> This is intended for e.g. background threads which read from a
>>>> database
>>>>>>> once per second and compare the old values with the new ones.
>>>>>>> If there was any difference then the set of attributes get reported
>>>> back
>>>>>>> to the Config (which in turn clears the caches, etc).
>>>>>>>
>>>>>>>
>>>>>>> Now where to put this method?
>>>>>>>
>>>>>>> My candidate would be
>>>>>>> org.apache.deltaspike.core.api.config.ConfigResolver.ConfigProvider
>>>>>>> I do not want to put it into the Config interface itself because it
>> is
>>>>> not
>>>>>>> a user contract thingy.
>>>>>>> And I also do not want to put it into ConfigResolver becasue I'd like
>>>> to
>>>>>>> have the impl only available internally and not bloat the
>>>> ConfigResolver
>>>>>>> any further.
>>>>>>>
>>>>>>> Wdyt?
>>>>>>>
>>>>>>> LieGrue,
>>>>>>> strub
>>>>>
>>>>>
>>>>
>>
>>