Is storing data that can be inferred go against data normalization?

To simplify lets say I have two tables PROJECTS and JOBS. A user creates a project and runs 1 or more jobs using a project.

So it would possible look something like this:

PROJECTS

  • [PK] project_id
  • etc.

JOB

  • [PK] job_id
  • [FK] project_id
  • job_run_date
  • etc.

Ideally I want to be able to list all projects by last used (last job run) and display them to the user. When I create a job, I can insert the job then update the project entry to have a last used; or I can just have it insert the job, then when I’m displaying all projects, I can do a join and aggregation with JOBS to get the last used date.

I figure, the drawback of the join is that as you have more jobs, performance may be slowly impacted over time to do the joins and aggregations. Whereas, being somewhat redundant increases the chance of a transaction failure and wastes a bit of space.

If I were to choose the update both tables (i.e. have a last_used` column in PROJECTS), would that be going against normalization (or good db design practices)?

Path normalization issue with semicolon in Tomcat

I have observed a path normalization issue in the tomcat when i was passing “..;” in the URL. I tested this out with Nginx and Apache-tomcat-10.0.0-M4. I was able to access file directories which are not allowed in the Nginx. Please find the below screenshots for more information,

  1. Nginx Configuration:

Nginx Configuration:

As per the above configuration i have enabled /app/ context path only in Nginx.

  1. I created two directories called App (contains test.html) and App2 (contains test2.html) in the Tomcat ROOT directory.

enter image description here

  1. As per the above Nginx configuration it allows access only to app/test.html. But using semicolon it is possible to access app2/test2.html file as well.

Normal behavior

enter image description here

Behavior with the semicolon

enter image description here

As per the above screenshot, it is allowed to access to the test2.html page via Nginx with semicolon even app2 context path is not define in the Nginx configuration. Also please note that i checked this behavior without the Nginx and it was noted the same behavior. I was able to reproduced this issue directly in the Tomcat 9.0.12 and Tomcat 10.0.0-M4.

enter image description here

enter image description here

Is this already a known issue? or is this the normal behavior in the Tomcat level? A Similar issue has discussed in Blckhat(See below link for more details).

custom post types, custom fields and normalization

I have decided to redesign a plug I developed to use custom post types and custom fields instead of custom tables in the database.

I have some questions about abstracting data into new post types. These questions are based on the assumption that I should essentially be trying to replicate a relational database.

It’s a dictionary feature for a language learning blog. Here are the tables from the original. Simplified for readability.

CREATE TABLE words (     ID int(),     word varchar(),     parent_id int() );  CREATE TABLE nouns (     word_id int(),     plural varchar(),     [...] );  CREATE TABLE verbs (     word_id int(),     pres_part varchar(),     past_part varchar(),     [...] );  CREATE TABLE definitions (     ID int(),     word_id int(),     definition varchar(),     [...] );  CREATE TABLE examples (     ID int(),     defintion_id int(),     example varchar(),     [...] );  

Words will obviously be the first custom post type.

I’ve decided to use custom fields for the noun and verb information as they are 1:1 relationships and only existed to reduce null entries in the words table. I believe WordPress will only create an entry in the postmeta table if the fields are actually used. So that would solve that problem.

Now the main question is; should I store definitions and examples as custom post types as I would in a relational database or just use ACF repeaters? These are both genuine 1-many relationships. Although a single definition will probably only ever have a maximum of 3 examples.

The reader might wish to search for keywords within the definitions and/or examples but they would never want to view a single definition as page.

Definitions and examples might however be used outside the context of their respective word in the future. With a flashcard app for example. Is it easy to uniquely identify and request a single entry from a repeater?

Normalization in IBM hexadecimal floating point

According to the Wikipedia link of IBM hexadecimal floating point:

Consider encoding the value −118.625 as an IBM single-precision floating-point value.

The value is negative, so the sign bit is 1.

The value 118.62510 in binary is 1110110.1012. This value is normalized by moving the radix point left four bits (one hexadecimal digit) at a time until the leftmost digit is zero, yielding 0.011101101012. The remaining rightmost digits are padded with zeros, yielding a 24-bit fraction of .0111 0110 1010 0000 0000 00002.

The normalized value moved the radix point two digits to the left, yielding a multiplier and exponent of 16+2. A bias of +64 is added to the exponent (+2), yielding +66, which is 100 00102.

Combining the sign, exponent plus bias, and normalized fraction produces this encoding:

S Exp Fraction
1 100 0010 0111 0110 1010 0000 0000 0000
In other words, the number represented is −0.76A00016 × 1666 − 64 = −0.4633789… × 16+2 = −118.625

Now, the definition of normalization according to Wikipedia says that

In base $ b$ a normalized number will have the form $ ±d_0.d_1d_2d_3…×b_n$ where $ d_0≠0$ , and the digits $ d_0,d_1,d_2,d_3,…$ are integers between $ 0$ and $ b−1$

So, how is $ 0.011101101012 \times 16^2$ a normalized number?
In fact this number cannot be represented as a normalized one with base $ 16$ exponent because the closest we can get is $ 1.1101101012 \times 16^1 \times 2^2$ . What am I missing here?

Can someone help me with database normalization (1NF, 2NF and 3NF)?

I am taking university classes in database management for a half of year now. Although, it does not give enough of information and it is not clear enough to understand databases well. Now I have a task:

In the table there are columns, that needs to have functional dependencies by normalization formulas. Write 1NF, 2NF and 3NF by the given information. enter image description here

If someone could help me with this table and explain what I need to do- that person would be a life saver, honestly.

Thanks in advance for any help! And sorry for the language mistakes- english is not my native.

zscore v.s. minmax normalization, why their results look the same

I’m trying to normalize my time series with two different normalization method, minmax and zscore and compare the results. Here is my code:

def scale_raw_data_zscore(raw_data):     scaled_zscore = pd.DataFrame()     idx = 514844     values = raw_data.loc[idx]['d_column'].values     values = values.reshape((len(values), 1))     scaler = StandardScaler()     scaler = scaler.fit(values)     normalized = scaler.transform(values)     normalized = normalized.reshape(normalized.shape[0])     normalized = pd.DataFrame(normalized, index=raw_data.loc[idx].index, columns=raw_data.columns)     scaled_zscore = scaled_zscore.append(normalized)     return scaled_zscore  def scale_raw_data_minmax(raw_data):     scaled_minmax = pd.DataFrame()     idx = 514844     values = raw_data.loc[idx]['d_column'].values     values = values.reshape((len(values), 1))     scaler = MinMaxScaler(feature_range=(0, 1))     scaler = scaler.fit(values)     normalized = scaler.transform(values)     normalized = normalized.reshape(normalized.shape[0])     normalized = pd.DataFrame(normalized, index=raw_data.loc[idx].index, columns=raw_data.columns)     scaled_minmax = scaled_minmax.append(normalized)     return scaled_minmax  def plot_data(raw_data, scaled_zscore, scaled_minmax):     fig = pyplot.figure()     idx = 514844     ax1 = fig.add_subplot(311)      ax2 = fig.add_subplot(312)      ax3 = fig.add_subplot(313)     raw_data.loc[idx].plot(kind='line', x='date', y='d_column', ax=ax1, title='ID: ' + str(idx), legend=False, figsize=(20, 5))     scaled_zscore.reset_index(drop=True).plot(kind='line', y='d_column', ax=ax2, title='zscore', color='green', legend=False, figsize=(20, 5))     scaled_minmax.reset_index(drop=True).plot(kind='line', y='d_column', ax=ax3, title='minmax', color='red', legend=False, figsize=(20, 5))     pyplot.show() 

I’m adding the plot of the results. Why the results of both scaling methods are exactly the same? And why they have a different pattern from the raw data?

enter image description here

How to implement Batch Normalization into DDPG?

So i got this DDPG implementation and i wanted to try to apply batch_normalization on it, but i can’t figure out how to do it, it seems to be really difficult?!

Could somebody help me out?

DPPG Code: https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow/blob/master/contents/9_Deep_Deterministic_Policy_Gradient_DDPG/DDPG_update2.py

Derivative of L2 Normalization?

I have the following vector C, which I want to do a dot product against T to minimize the error of that.

C is:

enter image description here

T is:

enter image description here

If I didn’t need a normalized version of C, the error would simply be

L = enter image description here

and the gradient of dL/dA for example would be just [Tx, Ty, Tz]

But if I need C to be normalized, it goes crazy, C then becomes:

enter image description here

and the error becomes:

enter image description here

At this point I don’t clearly know how to compute dL/dA anymore, as there are too many coupled variables. What do I do?