# Need program/batch file to edit date.txt file



## fscldhppr (Mar 29, 2010)

I have a txt file that, after entering the current/other date, I wish for it to be edited....either by use of a batch file or whatever means available in order to remove all lines in the text file in which the "entered date" does not fall into any variation of date/ranges of each specific line.

An example of the(to be edited)text file looks like this....

(I've included line #'s for reference...line #'s actually do not exist on the dates.txt).

1. Above 3/10 - 4/1, Exc. 3/31
2. Above 3/10 - 3/28
3. Above 3/10 - 4/2, Exc. 3/15 - 3/16, 3/22 - 3/23, 3/31
4. Above Disc. 4/30
5. Above Disc. 4/30, Exc. 3/13, 3/27
6. Above Disc. 4/4, Exc. 3/15 - 3/19, 3/22 - 3/26, 3/29 - 4/2
7. Above Eff. 3/11
8. Above Eff. 3/11, Exc. 3/12 - 3/17, 3/19 - 3/24, 3/26
9. Above Exc. 3/2
10. Above Exc. 3/2 - 3/10, 3/15, 4/1

The above date.txt file may contain several thousand lines...of various dates/ranges/exceptions....but all lines will only include those such variables listed above....such as a specific date range, discontinued date, exception date(s), or a combination of dates with exceptions(as shown in the example).

So bottom-line what I am hoping to do is find a method which allows me to enter...say a date of "4/1"...then have the batch file(whatever)read each line of the above date.txt file... and either remove completely(leaving space) or remove/replace with other text(such as "NOT VALID")each line in which the entered date(4/1) does not fall into ANY of the various date/exception/combinations listed for each line.

Using the above example...and using the applied date of "4/1"...the date.txt file would be edited to remove/rename line #'s 2,6,10(for reasons highlighted in red)...and simply save to the original file name(dates.txt).

Is this possible with a batch file or would it require another method.

Thanks


----------



## TheOutcaste (Aug 8, 2007)

Welcome to TSG!

Not too hard if you just want to match any date.
From your sample it looks like it follows these rules:
Two types of data, a Single Date(SD), or a Date Range(DR) (the Data Items).
Dates do not contain a leading 0, or the year.
Looks like you will have at least one Data Item, at most 4 Data Items on any one line.
If there is more than one data item, the data items are comma delimited.

One problem I can see is the dates don't include the year. Will the dates in the *data.txt* file ever be more than one year old?
Example, you are searching for 4/1; could the file contain a range from last year, say 3/30/2009-4/2/2009?


----------



## fscldhppr (Mar 29, 2010)

Hello...thanks for the reply

The data I work with is changed every couple of months, providing the year is not necessary as it is not important in how I use the information.

Edit: If inserting the year is necessary to provide a solution...that would not be a problem...as once the *invalid* date lines are removed and only the *valid* ones are left, what I do next is to remove ALL date lines...leaving only the "Above" information(from valid dates)intact.

Up until now I've just been using a text editor to remove the lines which(for whatever date I choose)do not apply. That takes a long time...and with sometimes thousands of lines to edit it is very easy to inadvertently remove a valid date line or leave one that is invalid.

As you can see from the example I posted...I have to remove not only the invalid(date)line, but also the line immediately above it. Not a problem using multi-line reg. expressions with a text editor...but tedious...with all the various date/ranges in question.

I'm just looking for a shortcut...seeing as I have to apply it every couple of months or so.


----------



## TheOutcaste (Aug 8, 2007)

fscldhppr said:


> remove completely(leaving space) or remove/replace with other text(such as "NOT VALID")each line in which the *entered date(4/1) does not fall into ANY* of the various date/exception/combinations listed for each line.


A single date is a range with the same start and end dates, so if the date you are searching on is not an exact match for that single date, the line should be removed, as it does not fall into that date range.

So following that rule you would remove these lines for the reasons shown:
2 - 4/1 *does not fall into* the range 3/10 - 3/28
4 - 4/1 *does not fall into* the range 4/30 - 4/30
5 - 4/1 *does not fall into* the range 4/30 - 4/30, 3/13 - 3/13, or 3/27 - 3/27
7 - 4/1 *does not fall into* the range 3/11 - 3/11
8 - 4/1 *does not fall into* the range 3/11 - 3/11, 3/12 - 3/17, 3/19 - 3/24, 3/26 - 3/26
9 - 4/1 *does not fall into* the range 3/2 - 3/2

6 - would not be removed as 4/1 _*is*_ in the 3/29 - 4/2 range
10 - would not be removed as 4/1 *is* in the 4/1 - 4/1 range.

There are 4 date "fields": *Above*, *Disc.*, *Eff.*, *Exc.*.

To remove just the lines you show, *2, 6, 10*, there should be four rules. If the first rule determines the line should be removed, the other rules are not processed.
Always remove line if search date is contained in any date in the *Exc.* field.
Remove line if search date is after the date range in *Above*
Remove line if search date is after the date in *Disc.*.
Do not remove line if search date is after the *Eff.* date.

Your sample doesn't show these possibilities:
A. Above Disc. 4/30, Exc. 4/1
B. Above Eff. 3/11, Exc. 4/1
C. Above Disc. 4/1
D. Above Eff. 4/1
E. Above 3/10 - 3/28, Disc. 4/30

Which of these should be removed? I'm guessing just A and B. C would be removed if the search date is 4/2 or later, D if it is 3/31 or earlier.
Would a line like E ever appear in the file? If so, which takes precedence, the *Above* date range, or the *Disc.* date?

To further define the fields and what they contain, this is how I think you are defining them:


*Above* is always one date range
Will not have a date range if *Disc.* or *Eff.* is present
May have no date range if *Exc.* is present
*Disc.* is one single date, the end date of a date range
Start date is the earlier of today or the date specified.
*Eff.* is one single date, the start date of a date range with no end date
*Exc.* can be a single date or a date range
More than one item is possible.
 Will *Disc.* or *Eff.* ever contain a date range, or have more than one date?
Will *Above* ever contain just a single date, or more than one date range?
Will *Above* ever contain a date when *Disc.* or *Eff.* are present?
Will *Disc.* or *Eff.* ever appear on the same line? If so, which takes precedence if *Exc.* does not force a removal?

Checking for multiple dates in any one field is trivial, but the precedence of the rules must be strictly defined.
Dates that span the end of the year are easy to check as well, just split the range into two before checking, so 12/20 - 1/10 becomes 12/20 - 12/31,1/1 - 1/10. Would only be a potential problem if the data covers an entire year.


----------



## fscldhppr (Mar 29, 2010)

Not sure if it will help, but the data in question is timetable data used by the airlines....the various effective dates obviously allow the customer to know which dates for a particular flights are valid.

A sample of the actual data(date.txt)I wish to edit is as follows...

(AAL) (CPH)
0620 0705 SK1228 M81 X7 0 N
Above Disc. 4/2
0620 0705 SK1228 M81 X67 0 N
Above 4/5 - 5/13
0640 0725 SK1202 319 67 0 N
Above Eff. 3/28
0645 0730 SK1228 319 7 0 N
Above Disc. 3/21
0705 0755 SK1202 321 X7 0 N
Above Disc. 4/1
0705 0755 SK1202 321 X67 0 N
Above 4/5 - 5/14
0820 0905 SK1206 M81 X67 0 N
Above Disc. 5/12, Exc. 3/31 - 4/5, 4/30
1050 1135 SK1208 CR9 67 0 N
1055 1140 SK1208 CR9 X67 0 N
Above Eff. 3/29
1100 1145 SK1208 M81 X67 0 N
Above Disc. 3/26
1325 1410 SK1214 CR9 6 0 N
Above Disc. 3/27
1330 1415 SK1214 M81 X6 0 N
Above Disc. 3/31
1330 1415 SK1214 M81 X17 0 N
Above 4/3 - 4/10
1330 1415 SK1214 M81 Daily 0 N
Above Eff. 4/11
1605 1650 SK1210 M81 Daily 0 N
1735 1820 SK1218 CR9 X6 0 N
Above Disc. 3/26
1735 1820 SK1218 CR9 X67 0 N
Above 4/5 - 4/29
1735 1820 SK1218 CR9 X67 0 N
Above 5/3 - 5/12
1740 1825 SK1218 CR9 7 0 N
Above Eff. 3/28
1840 1925 SK1204 M81 X67 0 N
1925 2010 SK1220 CR9 6 0 N
Above Exc. 4/3

I am wanting to remove all flights(AboveXXX)that would not be valid for a particular date....such as a ref. date of 4/1....in which case those(highlighted in red)would be removed...for various reasons(no longer valid, not valid yet, 4/1 falls within exception dates/ranges, etc.).

You are correct as to DATE fields, "Above, Disc., Eff., Exc."..while the date possibilities themselves are numerous, they all fall into the same 7 field patterns...

A. Above...(date range)
B. Above.......(date range),Exc....(one or more date ranges separated by comma)
C. Above Disc.....(date specific)
D. Above Disc...(date specific),Exc....(one or more date ranges separated by comma)
E. Above Eff....(date specific)
F. Above Eff...(date specific),Exc....(one or more date ranges separated by comma)
G. Above Exc...(date specific) or (one or more date ranges separated by comma)

The excerpt above is from my .txt file which is actually 173,241 lines long. Doing a search in that text file for all the different(Abovexxxxx)dates ......there are 4153 different lines...I've attached that so you can see the various dates patterns in question.

(now you can see why I'm looking for a shortcut )

You wrote...

_Your sample doesn't show these possibilities:
A. Above Disc. 4/30, Exc. 4/1
B. Above Eff. 3/11, Exc. 4/1
C. Above Disc. 4/1
D. Above Eff. 4/1
E. Above 3/10 - 3/28, Disc. 4/30

Which of these should be removed? I'm guessing just A and B. C would be removed if the search date is 4/2 or later, D if it is 3/31 or earlier....._

You are correct with A and B

C is good thru 4/1 so it remains, obviously any date before 4/1 would cause that line to be removed, any date after 4/1 would remain

D is effective 4/1...so it remains
_
Would a line like E ever appear in the file? _.....
No, just one of the 7 patterns above.


----------



## TheOutcaste (Aug 8, 2007)

Disc and Eff will need year info, or some sort of a rule for a minimum time period.
For example:
*Above Disc. 1/5*
Date to search with 12/15
1/5 always comes before 12/15 as far as the computer is concerned, as the year isn't specified, so the line would be removed.
If you specify a minimum 3 month time period, it would be considered like this:
*Above Disc. 9/1-1/5*
So searching for 12/5 would not remove the line.

This would be a minimum, so *Above Disc. 6/15* would be considered *Above Disc. 1/1-6/15*

You can use the same period for the Eff. dates as well, or a separate minimum.

The sample you attached has dates ranging from 3/1 to 5/16, so 1 month back and 1.5 months forward. If that's typical, a 3 month range should work.


----------



## fscldhppr (Mar 29, 2010)

The 3 month period for timetable data is pretty much max for most carriers, depending on the season. Many airlines belong to alliances which put out their own timetables(along with the individual carriers timetables), and those could be updated monthly. The data I've used for my examples is from work I'm doing on the Star Alliance timetable...

http://www.staralliance.com/assets/doc/en/services/tools-and-downloads/pdf/StarAlliance.pdf

Which is currently valid from 4/1 - 6/13, while OneWorld Alliance

http://www.trvlink.com/download/oneworld/oneworld.pdf

is only valid for a short period 4/2 - 4/30

The reference date I've been using (4/1) is actually pretty flexible in that, while I would like to keep the data as current as possible I am more interested in extracting the most accurate(least amount of errors to correct)information from the timetable....that might mean trying different dates within the valid timetable period. The reason for trying different ref. dates is that the timetable data can itself be fraught with errors ....meaning that it may show flights valid for a certain date period....when actually they are not(due to typo's, miss-print, etc.).

I have found by trying different ref. dates I can sort of "find a sweet spot" that has less errors and requires less correcting.

By having a method whereby I can test different(individual)ref. dates by just entering the date and letting it(batch, script, etc.) delete the non-valid flights...this will allow me find the most error-free data.

My current method is to pick a date within the timetable period(usually the beginning...4/1 in the case of Star Alliance)and eliminate all flights that are not valid for that particular date by way of a text editor(Ps Pad). As you can imagine that takes quite a while, especially when dealing with large amounts of data such as you get with a multiple carrier/alliance timetable. And if I'm not careful I can inadvertently take out valid data or leave invalid data....which, when found requires me to go back to the physical timetable, other outlets to correct.


----------



## TheOutcaste (Aug 8, 2007)

OK, give this a try. It will create two files, a new one with the lines removed leaving the ones you want, and another with the lines that were removed.
If you wish to overwrite the original, you can uncomment the Move statement in line 74.

Not sure how slow this will be, hard to extrapolate;ate from a 40 line sample to the 173,000+ size files you are using, but it takes .56 seconds for the 40 lines, which extrapolates to 40 minutes. You may want to try it out on a smaller sample of 2-5000 lines first to get an idea of the speed. I haven't really looked at seeing if there was a faster way to do things, just kind of pieced things together.


```
@Echo Off
SetLocal EnableDelayedExpansion
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Edit the next 7 lines as needed
:: The two MinPeriod values are in months and set the minimum time frame
:: This allows the program to function orrectly when dates span the end of the year
:: _DataFile is the path to the source file
:: _NewDFile is the path to the New file after lines are removed
:: _RemFile is the path to the file that will contain the lines that are removed
:: The two If Exist statements delete these files each time it's run.
:: Rename the files if you wish to keep them before running again
:: Another option is to add the date and or time to the file name
:: Example, to add the time, use [path]ndate%time%.txt
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Set _DiscMinPeriod=3
Set _EffMinPeriod=3
Set _DataFile=C:\Test\date.txt
Set _NewDFile=C:\Test\newdate.txt
Set _RemFile=C:\Test\Removeddate.txt
If Exist "%_NewDFile%" Del "%_NewDFile%"
If Exist "%_RemFile%" Del "%_RemFile%"
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
Set _Valid=00312931303130313130313031
Cls
:_Ask1
Echo.
Echo Press Enter to Exit.
Set _SearchDate=
Set /P _SearchDate=Enter date to search for - 
If Not Defined _SearchDate Goto :EOF
For /F "Tokens=1,2 Delims=/ " %%I In ('Echo %_SearchDate%') Do Set /A _SMonth=%%I&Set /A _SDay=%%J
:: Remove leading zero if present
If 1%_SMonth% LSS 110 Set _SMonth=%_SMonth:~-1%
If %_SMonth% GTR 0 If %_SMonth% LSS 13 Goto _ChkDay
:_DateErr
Echo %_SearchDate% is not a valid date. Please Re-enter&Echo.&Goto _Ask1
:_ChkDay
If 1%_SDay% LSS 110 Set _SDay=%_SDay:~-1%
Set /A _tmp=_SMonth*2
Set _Dmax=!_Valid:~%_tmp%,2!
If %_SDay% GTR 0 If %_SDay% LEQ %_Dmax% Goto _DateOK
Goto _DateErr
:_DateOK
Set /A _SearchDate=_SMonth * 100 + _SDay
Set _PrevLine=
:: Process Data
:: Split the line into Fields by calling Extract
:: Then check each of the fields for a match and set Remove if line is to be removed
@For /F "Tokens=* Delims=" %%I In ('Type "%_DataFile%"') Do (
  Set _CurLine=%%I
  If /I "!_CurLine:~0,5!"=="Above" (
    Call :_Extract "%%I"
    Set _Remove=0
    If NOT "!_Exc!"=="" Call :_Parse !_Exc!&If !_Match!==1 Set _Remove=1
    If NOT !_Above!==0 Call :_Parse !_Above!&If !_Match!==0 Set _Remove=1
    If !_Remove!==0 (
     If NOT "!_Disc!"=="" Call :_Parse !_Disc!&If !_Match!==0 Set _Remove=1
     If NOT "!_Eff!"=="" Call :_Parse !_Eff!&If !_Match!==0 Set _Remove=1
    )
    If !_Remove!==1 (
      >>"%_RemFile%" Echo.!_PrevLine!
      >>"%_RemFile%" Echo.%%I
    ) Else (
      >>"%_NewDFile%" Echo.!_PrevLine!
      >>"%_NewDFile%" Echo.%%I
    )
    Set _PrevLine=
  ) Else (
    If NOT "!_PrevLine!"=="" >>"%_NewDFile%" Echo.!_PrevLine!
    Set _PrevLine=%%I
  )
)
:: Uncomment this line to replace the original Source file
::>Nul Move /Y "%_NewDFile%" "%_DataFile%"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
::                  Subroutines
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Separates the different fields and assigns them to variables
:: First see if Above contains a date, if not put in zero
:: Then split the line into the variables.
:_Extract
Set [email protected]
Set _Disc=
Set _Eff=
Set _Exc=
Set _DataLine=%~1
:: Remove Above and spaces from line
Set _DataLine=%_DataLine:~6%
Set _DataLine=%_DataLine: =%
Set _DataLine=%_DataLine:[email protected]%
Set _DataLine=%_DataLine:[email protected]%
Set _DataLine=%_DataLine:[email protected]%
If "%_DataLine:~1,1%"=="D" Set [email protected]
If "%_DataLine:~1,1%"=="E" Set [email protected]
If "%_DataLine:~1,1%"=="X" Set [email protected]
Set _DataLine=%_Above%%_DataLine%
Set _Above=
For /F "Tokens=1-4 [email protected]" %%I In ("%_DataLine%") Do Call :_Split "%%I" "%%J" "%%K" "%%L"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Determines which fied each parameter belongs in based on the first letter
:_Split
Set _P1=%~1
Set _P2=%~2
Set _P3=%~3
Set _P4=%~4
If NOT "%_P1%"=="" Call :%_P1:~0,1% "%_P1:~1%"
If NOT "%_P2%"=="" Call :%_P2:~0,1% "%_P2:~1%"
If NOT "%_P3%"=="" Call :%_P3:~0,1% "%_P3:~1%"
If NOT "%_P4%"=="" Call :%_P4:~0,1% "%_P4:~1%"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Assign data to variables. Disc and Eff are Converted
:: to ranges with a minimum length in months of _DiscMinPeriod
:: and _EffMinPeriod. Ex, with a 3 month minimum, if Desc
:: date is Jan, it will become a range of Sept=Jan.
:A
Set _Above=%~1
Goto :EOF
:D
Set _Disc=%~1
Set _Disc=%_Disc:/=,%
Call :_Convert %_Disc%
Set _Disc=%_P1%
Set /A _P2=_P1/100-_EffMinPeriod
If %_P2% GTR 0 (Set _P2=1) Else Set /A _P2=_P2+12
Set _Disc=%_P2%/1-%~1
Goto :EOF
:E
Set _Eff=%~1
Set _Eff=%_Eff:/=,%
Call :_Convert %_Eff%
Set _Eff=%_P1%
Set /A _P2=_P1/100+_MinRange
If %_P2% LSS 13 (Set _P2=12) Else Set /A _P2=_P2-12
Set _Eff=%~1-%_P2%/31
Goto :EOF
:X
Set _Exc=%~1
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: All fields are processed here. Each parameter except
:: Exc will be a single date range, Exc can be multiple
:: date ranges, or a single date. The - and / are removed
:: Then another call is made to separare the month and date.
:: These are converted to numbers for comparison.
:: Result is returned in the _Match variable
::          0 = SearchDate is not in the range
::          1 = SearchDate is in the range
:_Parse
Set _Match=0
If "%1"=="" Goto :EOF
Set _Date=%1
:: Replace - and / with , so month and day will be two parameters
:: a range will be four
Set _Date=%_Date:-=,%
Set _Date=%_Date:/=,%
Call :_Check %_Date%
If %_Match%==1 Goto :EOF
Shift
Goto _Parse
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:_Check
:: Will be two or four parameters
Set /A _P1=%1*100+%2
:: If Date Range, calc end date, set to 0 if not range
If "%~3"=="" (Set _P2=0) Else (Set /A _P2=%3*100+%4)
:: if Range doesn't span end of year check if search date is within the range.
:: If Range spans end of year, check if date between start and 12/31 and 1/1 and end
If "%~3"=="" (
:: Single Date
  If %_P1%==%_SearchDate% Set _Match=1
) Else (
  If %_P2% GEQ %_P1% (
    If %_SearchDate% GEQ %_P1% If %_SearchDate% LEQ %_P2% Set _Match=1
  ) Else (
    If %_SearchDate% GEQ %_P1% If %_SearchDate% LEQ 1231 Set _Match=1
    If %_SearchDate% GEQ 101 If %_SearchDate% LEQ %_P2% Set _Match=1
  )
)
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Convert a date or date range to one or two numbers
:: Number= Month*100+day, so number is mmDD
:: Will be two or four parameters
:_Convert
Set /A _P1=%1*100+%2
Goto :EOF
```


----------



## fscldhppr (Mar 29, 2010)

It works!

Just finished testing it on the long(170k lines)version....wish I would have timed it, but it honestly didn't take that long to complete.

I also completed the process of configuring the new data.txt into the format necessary to use a program I have for compiling the data for use. 

I suppose I should have mentioned it, but I use the data to create real-time airline traffic for Microsoft Flight Simulator. I share those flight plans with other Flight Sim enthusiasts on various FS websites.

Your work alleviates one of the most frustrating steps in putting together all the data for use...and will definitely shorten the whole process.

While I am still checking out the (large)new output file, so far the only errors I am coming up with are the usual suspects, those are all mistakes in the timetable itself.

Anyway, I cannot thank you enough for using your time and your talents on this.

Just as a side note...on one of the Flight Sim boards I frequent was someone who just lost 4 friends(accident?), he was looking for newer plans for US Airways(which are one of the carriers on the data.txt). I had pretty much stopped creating flight plans, as the process took up too much of my time, but after reading that I decided that maybe I could give the person a little solace by putting USAirway's together. Was looking at a couple of weeks to sort/work out the data....but instead decided to ask for help in reducing the workload. Which is how I made it to this website.

Again, I appreciate your help.


----------



## Squashman (Apr 4, 2003)

MS gave me Flight Simulator a few yeas ago but I never loaded it up. How does that integrate with it?

If you want to know how long it took, look at the file time stamps. Created VS modified.


----------



## fscldhppr (Mar 29, 2010)

Flight Simulation has grown in innovation by leaps and bounds over the last three releases. I still remember the first one I bought for my Atari 400...basically PONG with wings. The third party developers and their creations have pretty much turned a game into an addiction...for some anyway.

...below is an excerpt from the readme of one of the programs that came out a while back which allowed for real-time AI creation...

With Flight Simulator 2002 Microsoft introduced a new concept called AI traffic. AI traffic is actually other aircraft flying around in Flight Simulator. They are controlled by the AI traffic and ATC engine in Flight Simulator. Originally it was only the default aircraft in Flight Simulator that were flying AI traffic.

A few months after FS2002 came out, a developer called Lee Swordy, made a program called Traffic Tools (TTools for short). This program enabled users to customize the AI traffic in Flight Simulator. TTools opened a new dimension to Flight Simulator. It allowed users to populate their Flight Simulator world with real life airline traffic.

The Most Realistic AI (or MRAI for short) is a project that started in September 2002. It was founded by Arnt Helge Haaland and Vidar Haaland. With the capabilities TTools gave us, our goal was to set a new standard and make the best AI flightplans available for Microsoft Flight Simulator 2002/2004.

TTools and MRAI Compiler became the tools of choice to disassemble, re-assemble AI flight plans.

Other programs/tools were created for modifying airports(AFCAD), aircraft models, panels, sounds, etc.....it really is insane how much FS advanced in 10+ years.

(some people who own FSX might differ with that "advanced" opinion...seems it really taxed system resources....and many in the hobby shelved it shortly after purchase, while others have been warming up to it with advanced/cheaper hardware... I use fs2004 myself.)

Flight Simulation is one of those things that people(aviation enthusiasts)really get into, while others(shoot-em-up gamers)get completely bored with....real quick.


----------



## fscldhppr (Mar 29, 2010)

Oh BTW, one of the output files was modified by me later on so the time stamp was off on that one....while the other showed 28 min 32 sec between created Vs modified....if I recall correctly that seems about right.

But I did have other programs/browser open at the time.

If you want to see what AI looks like, there are several videos on YouTube which show what real-time traffic looks like in FS. Just search for "AI traffic" for either fs2004 or fsx.


----------



## TheOutcaste (Aug 8, 2007)

Glad it's working for you. Better times than the estimate, but I was hoping it would be under 20 minutes. Have to take a 2nd look in a day or so, see if something comes to mind that might speed it up.

I used to spend way too much time with MS Flight Simulator. That was on a 486DX33. Still have the two 3.5" floppies, Version 5, Copyright 1993. It was 11 years old back then; the first MS version came out in 1982. It's a shame they've closed that division down.

It's certainly changed since then. I could easily see myself getting the whole virtual cockpit, but then I'd never get anything done.


----------



## fscldhppr (Mar 29, 2010)

Same here, I pretty much owned them all. Not sure if you remember Sublogic ATP, it was actually pretty good for it's time(3d glasses and all).

I too was sorry to see ACE's(FS team)being let go. With all the complaints surrounding FSX(not to mention vista)you had to know somebody was going to get hurt. I have never owned FSX so not sure what all the fuss was about....just heard about resource hog issues....though it seems folks (with newer hardware) are now enjoying it.

BTW, speaking of virtual cockpits...you should check out the stuff from Project Magenta...like you said...to have that kind of equipment in a full motion simulator...I too would never leave the house.

(Still...a full view virtual reality headset would be nice)


----------



## Squashman (Apr 4, 2003)

I have FS X. Maybe I will loaded it up on my wife's new Windows 7 machine.


----------



## fscldhppr (Mar 29, 2010)

If you decide to load it up...check out http://www.world-of-ai.com/http://www.world-of-ai.com/allpackages.php

They have complete AI packages which will automatically install the aircraft & flight plans.

Of course some of the flight plans may be a little outdated....(that's where guys like me step in : )

Avsim, Flightsim are also good sources for newer flight plans...as well as a treasure chest of aircraft, panels, scenery....


----------



## fscldhppr (Mar 29, 2010)

TheOutcaste,

That was a great idea with the 3 file output(date, removedate, newdate).

I've been going through each file to look for any discrepancies after running the 170k line date.txt. The newdate.txt created appears to be spot-on. But I did find some on the Removedate.txt that appear to be valid date periods(using 4/1)that were removed(see attachment for all the valid date periods I pulled from that file).

My fault in that I don't believe this was one of the parameters/patterns we discussed earlier when you were putting it together. Sorry about that.

Anyway, there's no timetable(no pun intended)on this....if you're busy with other projects, I certainly understand...just thought I'd let you know how the larger file worked out.

Thanks


----------



## TheOutcaste (Aug 8, 2007)

OK, found the little bug that slipped by. Forgot to remove the comma after the *Eff* date if there was one, so it wasn't checking the *Eff* date properly when there was anything after it.

Need to add one line in the subroutine for the *Eff* dates, at line 138
*Set _Eff=%_Eff:,=%*

So that section will look like this with the new line added:

```
:E
Set _Eff=%~1
Set _Eff=%_Eff:/=,%
Call :_Convert %_Eff%
Set _Eff=%_P1%
Set /A _P2=_P1/100+_MinRange
If %_P2% LSS 13 (Set _P2=12) Else Set /A _P2=_P2-12
Set _Eff=%~1-%_P2%/31
[COLOR=Red]Set _Eff=%_Eff:,=%[/COLOR]
Goto :EOF
```
See if that stops it from removing lines it shouldn't.


----------



## fscldhppr (Mar 29, 2010)

Going through the files now...looks like that solved it...Thank You!

(now if only Star Alliance wouldn't have so many errors on their PDF timetable : (

I wonder if their electronic timetable(for PDA, desktop, other devices)data is equally error prone. I downloaded and installed their desktop timetable application in hopes of finding the data file used for schedules in the app's folder. There are a couple of unknown file types in there that I suspect are used as the data files...but a.)they most likely are proprietary to their particular application and thus b.) unable to view information within.

The airlines appear to be slowly moving towards doing away with the printable PDF timetable in favor of electronic device applications....guess I better make use while I still can.

Anyway, thanks again for all your work on this project.


----------



## TheOutcaste (Aug 8, 2007)

Glad to hear it, and you're welcome!

Open Notepad (or better yet, Notepad++ or Notetab Light, they both will deal with non-text files better than Notepad will) and drag and drop one of the "unknown" files on it. Sometimes those unknown file types are really plain text, or may have the data you want in plain text. You'd probably have to copy it out of the file manually in the latter case, as batch files don't do well trying to grab text out of a binary file. A VBScript might work in that case though.


----------



## fscldhppr (Mar 29, 2010)

Tried the 2(suspected)data files, one has a .seg(compressed file?) extension, the other with a .fli ext,...as well as other files in the Star Alliance folder(with Notepad++), unfortunately just unintelligible text(coded/encrypted???).

(Notepad++...nice editor BTW)

I know in the past I attempted to(crack)the information on those files using other text/hex editors....all with no success.

Chances are the data itself would be identical to the pdf data....so nothing gained there(same errors).

I was also curious though how the information might be layed out in the file, in that the structure might differ from the pdf in such a way that would allow me a quicker path to create the "date.txt" file.

Currently(after copy/paste from the PDF timetable) I use PsPad with various regular expressions, then use excel to move columns of information in order to create the date.txt...I am a novice(at best)there also...but I'm learning a bit more each time I work with it. Eventually would like to build scripts for each of the individual airlines(timetable)formats, in order to speed up the process further....and use that time instead weeding out the errors in the timetable itself.

It would be almost like a full-time job to stay current with airline schedules, with all the constant changes in the industry(mergers, cutbacks, retiring aircraft, etc.). I usually only create flight plans for fall/winter and spring/summer....that's... "as real as it gets"....for me anyway. : )


----------



## Squashman (Apr 4, 2003)

Did you perhaps see this post from last year. He was working on an app to parse the pdf into a csv.
http://nothing.tmtm.com/2009/05/more-bmi-hacking/


----------



## fscldhppr (Mar 29, 2010)

Thanks for the link, no I had not seen that, even after searching the web for something similar a few years ago.

That is pretty much what I end up with, once I get all text editor stuff out of the way and turn it into a .csv for final editing. 

The final .txt ends up as...

SWA;211;ALB;BWI;6:30AM;7:55AM;12345;737

Which is basically the /Airline/Flt.#/Dep. time/Arr. time/Days of the week flight is active/Aircraft type used.

Heh, I figure with my luck, by the time I figure out all the shortcuts and such to eliminate all the time consuming tasks involved in putting flight plans together...the airlines will dump their printable(pdf)timetables altogether and will go exclusively with(encrypted data) hand-held device app's, etc.

Most of us who created real-time traffic, up until recently, got our data from the Fed Travel website. There the timetable data was converted to a pdb file which you could download for pretty much every airline around the world. Someone created a program(PDBReader)which would read/decipher/put into the proper format all the parameters required to build the flight plans. This made flight plan building so much easier than building from scratch.

Unfortunately(as of last year?)Fed Travel no longer makes(pdb file)timetable data available...so most of us are stuck with doing things the old way...and until we can find better source for timetable data(pay for/travel data info)..then, for me anyway, the pdf is all I have to work with. Once it's phased out...that'll pretty much do it for building a real-time FS traffic environment....for me anyway.


----------



## Squashman (Apr 4, 2003)

So what about this. Does this do the same thing you are trying to do?
http://www.airnavsystems.com/FS/


----------



## fscldhppr (Mar 29, 2010)

Yeah, I've heard of them. Actually seeing as they compile new timetable data daily they're are definitely over and above what I do(heh, wish I could get direct FAA data). I have heard mixed reviews(so-so aircraft models used, etc.)...but have never tried it myself. I get discount offers occasionally via email from them, so sometime in the past I must have looked into it(I believe it used to be more than $30 for a 3 month subscription???).

There are other commercial products, like Ultimate Traffic, that pretty much do the same in that they include the whole package of flight plans, aircraft, afcad's(airports upgraded to handle the real world traffic load)like AirNav....they just don't update their flight plans(daily??)like AirNav.

I guess I'm from the old school class of Flight Simulator, back in the early days there were some very creative folks who would share their ideas with others and create a whole slew of excellent freeware, from aircraft to scenery to whatever the imagination could come up with to keep the hobby interesting...not just for themselves but for new enthusiasts who were always welcomed and encouraged to join in the fun of making it "as real as it gets".

While I definitely don't fault those who have (somewhat) commercialized Flight Simulator(I have purchased more than a few excellent products myself) a lot has changed since the early days. When people started purchasing products the demand for perfection started showing up more and more on the forums...eventually it bled into the freeware market which frustrated and eventually drove out a lot of very good designers.

The few(modest)things I am able to contribute to Flight Simulation I do out of the enjoyment of the hobby. If I feel it is good enough to share with others I do so...if anyone has constructive criticism I will gladly fix/improve it for them....if they wish to become belligerent...then(as a freeware developer)I ask them if they want their money back : ) 

I would imagine you guys here have seen a few of those types....

I don't put nearly as much time into it as I used to, trying to keep up with all the constant changes in real-world commercial aviation is a challenge....which(I suppose)by itself keeps the hobby interesting.

But with time constraints being what they are today....one can only dedicate so much time.


----------



## Squashman (Apr 4, 2003)

Maybe the Airlines saw fit to make some money from this so they are making it harder to get their time tables in a usable format.


----------



## fscldhppr (Mar 29, 2010)

No doubt the airlines have been cutting expenditures to the bone and looking for revenue wherever they can find it(Spirit announced a $45 fee for "carry-on" luggage). Cutting redundant reservation/timetable outlets will no doubt be part of their streamlining.

For a funny look at airline cost cutting to the extreme..checkout former American Airlines CEO Bob Crandall...


----------



## Cookiegal (Aug 27, 2003)

Thread reopened as requested.


----------



## fscldhppr (Mar 29, 2010)

Thanks for re-opening the thread....

The OutCaste/Squashman.....

I know it's been a while, but I decided to load up flight simulator, and also try and create some AI( flight plans) for the winter season to share with other flight simmers using data extracted from United Airlines timetable.

The OutCaste, I've used your batch file with great success on most other data from the various carriers PDF timetables that I've tried it on. United's data has very often required quite a bit of error correcting after completion for some reason.....I just passed it off as mistakes on the timetable itself(the usual case). However, after parsing the data the way I normally do and running the date.txt.... I checked the Removeddate.txt, and noticed a few flights were deleted....that were actually valid for the set date period(1 week).

As an example of some of the misplaced data: Using the set date of 12/17, and running the date.txt with the parameters built in I should end up with all flights within the date period of 12/17-12/23....and for the most part that is the case. The following few flights were taken from the Removedate.txt....but should have remained and included in the newdate.txt. And, as before, they appear to fall under the same common pattern.....

1512 1742 UA3312? 738 6 0 G 2:30
*Above 12/18 - 12/31*
2315 0611 UA991 763 0 DS 11:56
*Above 12/21 - 12/31*
0645 0844 UA4287? ERJ X67 0 G 1:59
*Above 12/21 - 12/31*
1040 1110 UA5917? ER4 7 0 1:30
*Above 12/19 - 12/31*
1843 1920 UA6945? CRJ 7 0 1:37
*Above 12/19 - 12/31*
0848 1018 UA551 319 7 0 2:30
*Above 12/19 - 12/31*

As before, I'm not sure if this is a pattern that I neglected to provide initially, or perhaps it is one that cannot be included.

Anyway, worse comes to worse, I can always just go through the Removedate.txt and look for those valid flights, and just add those flight back into the newdate.txt.

Just thought I'd ask if perhaps there might be another parameter that could be added which can provide a fix......

Thanks again for your help guys...this has really been a time saver.


----------



## TheOutcaste (Aug 8, 2007)

fscldhppr said:


> remove all lines in the text file in which the "entered date" *does not fall* into any variation of date/ranges of each specific line.





fscldhppr said:


> 1512 1742 UA3312? 738 6 0 G 2:30
> *Above 12/18 - 12/31*
> 2315 0611 UA991 763 0 DS 11:56
> *Above 12/21 - 12/31*
> ...


If you enter 12/17, it should remove all date ranges that are not valid on 12/17. None of the above include 12/17, they all start after 12/17.
The first one begins 12/18, so is not valid for 12/17.
I don't see any mention of finding valid dates for a range in the thread, i.e. "Date entered through Date entered+X", only the specific date entered.


----------



## fscldhppr (Mar 29, 2010)

Hello again TheOutCaste...thanks for the reply.

You are correct. I think where I went wrong is that most timetable data(not all)are based on 1-week scheduling. So technically speaking the beginning dates of those mentioned above do fall into that category, that is to say that using the ref. date of 12/17 would include flights that commence on any period within that week of 12/17-12/23 and therefore would be valid.

As I mentioned, I can use my text editor to go through the Removeddate.txt, look for those that fall into the above category, and simply add those to the newdate.txt. Was just wondering if it was possible to add (yet another) parameter/variable that takes into account weekly based scheduling.

Fortunately many airline's pdf timetables aren't so radical with date periods....and the batch file works great on them.....unfortunately United's is all over the map with their date scheduling(especially during peak periods). Which is why doing those type of timetables would(in the past)take weeks(if not months)to sift through all the variables and enter the data manually into an excel file....and that just being one step in the process.

Obviously your work cuts down (that particular process) time considerably.

I suppose I should have asked earlier, but the file you created, is this considered a batch file, batch script, or something else? Just wanting to know, as perhaps I should start studying up on the language myself, for no other reason than so that I might be able to do my own work on future projects.

Thanks again for your time.


----------



## TheOutcaste (Aug 8, 2007)

Should be able to add it. Just a case of calculating an end date, then checking the first date in the Above/Effective date ranges to see if they are on or before that end date.
If the Eff date were 12/20, you'd want that line included as well I'd assume.

Did find a couple of errors where the wrong variable is being used. One is more a typo, the variables have the same value. The other is using a variable that doesn't exist, so would affect how it calculates effective dates. Instead of using a three month range for the effective date, it will only be to the end of the current month. This would only be a problem at the end of the year, so probably hasn't shown up yet.
Example, eff date of 12/30, search date of 01/01, it would not be included.

Change these two lines:
Line 126 in the* :D* subroutine
*Set /A _P2=_P1/100-_EffMinPeriod*
should be
*Set /A _P2=_P1/100-_DiscMinPeriod*

Line 135 in the *:E* subroutine
*Set /A _P2=_P1/100+_MinRange*
should be 
*Set /A _P2=_P1/100+_EffMinPeriod*

This would be a batch file or batch script, either term applies.
I should probably sit down and do this in VBScript, be good practice. And should be much faster, often by a factor of 10 or 100 if not more.

I'll see if I can get the 1 week part working first. Shouldn't take too long, probably this evening.

Would you want to Specify when to use that, always be X number of days, or enter a start and end date?


----------



## fscldhppr (Mar 29, 2010)

_Would you want to Specify when to use that, always be X number of days, or enter a start and end date?

_Being able to specify a start/end date would probably be the best option, seeing as not all carriers use the same date pattern, but either one would work just fine for pretty much all the U.S. based carriers that I use.

_.....Example, eff date of 12/30, search date of 01/01, it would not be included._

I remember your mentioning the problem earlier regarding the shift from Dec.(12) to Jan.(1). My work around for that(at least I think it worked)was to change those dates beyond Dec. 31....to Dec 31....before running your file. Seeing as I don't usually change flight plans anything more than 3-4 times per year, that works out OK for my use, that is, those flight plans are current enough.

_I should probably sit down and do this in VBScript_

Someone else over on the AIG(Alpha India Groups)board mentioned something about using VBScript on the whole STAR alliance timetable....though I must admit, I've actually had pretty good success using your file on that one.

I am currently trying to partner with a couple others from that forum to try and put out flight plans to the flight sim community a bit faster....I'm sure they would join me in saying "thank you" for your work on this.

The only other question I forgot to ask, that being Delta's timetable, which neglects to use either a 24:00 time standard, or use AM/PM to distinguish between am/pm flights, and instead uses *BOLD* type for PM times, which of course gets wiped out on cut/paste.... and thus cannot distinguish between AM/PM flight times....this makes parsing their information pretty much useless.

Can you see any type of work around for that? I looked at perhaps trying to convert the pdf(directly from the website) to a different format(html) in order to break it down and edit the thing....but not even sure how/if that is possible.

A link to their pdf timetable http://www.delta.com/schedules/travel/reservations/flight_sched/index.jsp
.....but it appears to be down right now.

Anyway, just thought I'd ask.


----------



## TheOutcaste (Aug 8, 2007)

I'll have to look at one of the PDFs, see if maybe there's an easy way to read it to get date info.

Give this version a try. Tweaked a few things so it should work better around the end of the year, and added a new variable, *_ValidPeriod*. Set this to 7 and it will include dates valid for the week starting with the Search date. Set it less than 2 and it's the old behavior. I haven't tested with larger values.

```
@Echo Off
SetLocal EnableDelayedExpansion
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Edit these lines as needed
:: The two MinPeriod values are in months and set the minimum time frame
:: This allows the program to function correctly when dates span the end of the year
:: _DiscMinPeriod and _EffMinPeriod: These are used to convert the Effective and Discontinue
:: Dates to a date range. This is specified in Months. Just makes it easier to check
:: _ValidPeriod is range to check. Use 7 for one week. Search on 12/1 would include 12/1-12/6
:: _Src is the Folder that contains the files, saves retyping the next 3 lines if you change it
:: _DataFile is the path to the source file
:: _NewDFile is the path to the New file after lines are removed
:: _RemFile is the path to the file that will contain the lines that are removed
:: The two If Exist statements delete these files each time it's run.
:: Rename the files if you wish to keep them before running again
:: Another option is to add the date and or time to the file name
:: Example, to add the time, use [path]ndate%time::=%.txt
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Set _DiscMinPeriod=3
Set _EffMinPeriod=3
Set _ValidPeriod=7
Set _Src=C:\Test\
Set _DataFile=%_Src%date.txt
Set _NewDFile=%_Src%ndate.txt
Set _RemFile=%_Src%Remdate.txt
If Exist "%_NewDFile%" Del "%_NewDFile%"
If Exist "%_RemFile%" Del "%_RemFile%"
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
Set _Valid=00312931303130313130313031
Set _VP=2
If %_ValidPeriod% LSS 2 (Set _VP=1) Else Set _VP=2
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
Cls
:_Ask1
Echo.
Echo Press Enter to Exit.
Set _SearchDate=
Set /P _SearchDate=Enter date to search for - 
If Not Defined _SearchDate Goto :EOF
For /F "Tokens=1,2 Delims=/ " %%I In ('Echo %_SearchDate%') Do Set /A _SMonth=%%I&Set /A _SDay=%%J
:: Remove leading zero if present
If 1%_SMonth% LSS 110 Set _SMonth=%_SMonth:~-1%
If %_SMonth% GTR 0 If %_SMonth% LSS 13 Goto _ChkDay
:_DateErr
Echo %_SearchDate% is not a valid date. Please Re-enter&Echo.&Goto _Ask1
:_ChkDay
If 1%_SDay% LSS 110 Set _SDay=%_SDay:~-1%
Set /A _tmp=_SMonth*2
Set _Dmax=!_Valid:~%_tmp%,2!
If %_SDay% GTR 0 If %_SDay% LEQ %_Dmax% Goto _DateOK
Goto _DateErr
:_DateOK
Set /A _SearchDate=_SMonth * 100 + _SDay
:: Calculate ending date for search
Set _Emonth=%_SMonth%
Set /A _Eday=_SDay + _ValidPeriod - 1
If %_Eday% GTR %_Dmax% (
  Set /A _Eday=_Eday - _Dmax
  If %_Emonth%==12 (Set _Emonth=1) Else Set /A _Emonth=_Emonth+1
)
Set /A _EndDate=_EMonth * 100 + _EDay
Set _PrevLine=
:: Process Data
:: Split the line into Fields by calling Extract
:: Then check each of the fields for a match and set Remove if line is to be removed
@For /F "Tokens=* Delims=" %%I In ('Type "%_DataFile%"') Do (
  Set _CurLine=%%I
  If /I "!_CurLine:~0,5!"=="Above" (
    Call :_Extract "%%I"
    Set _Remove=0
    If NOT "!_Exc!"=="" Call :_Parse1 !_Exc!&If !_Match!==1 Set _Remove=1
    If NOT !_Above!==0 Call :_Parse%_VP% !_Above!&If !_Match!==0 Set _Remove=1
    If !_Remove!==0 (
     If NOT "!_Disc!"=="" Call :_Parse1 !_Disc!&If !_Match!==0 Set _Remove=1
     If NOT "!_Eff!"=="" Call :_Parse%_VP% !_Eff!&If !_Match!==0 Set _Remove=1
    )
    If !_Remove!==1 (
      >>"%_RemFile%" Echo.!_PrevLine!
      >>"%_RemFile%" Echo.%%I
    ) Else (
      >>"%_NewDFile%" Echo.!_PrevLine!
      >>"%_NewDFile%" Echo.%%I
    )
    Set _PrevLine=
  ) Else (
    If NOT "!_PrevLine!"=="" >>"%_NewDFile%" Echo.!_PrevLine!
    Set _PrevLine=%%I
  )
)
:: Uncomment this line to replace the original Source file
::>Nul Move /Y "%_NewDFile%" "%_DataFile%"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
::                  Subroutines
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Separates the different fields and assigns them to variables
:: First see if Above contains a date, if not put in zero
:: Then split the line into the variables.
:_Extract
Set [email protected]
Set _Disc=
Set _Eff=
Set _Exc=
Set _DataLine=%~1
:: Remove Above and spaces from line
Set _DataLine=%_DataLine:~6%
Set _DataLine=%_DataLine: =%
Set _DataLine=%_DataLine:[email protected]%
Set _DataLine=%_DataLine:[email protected]%
Set _DataLine=%_DataLine:[email protected]%
If "%_DataLine:~1,1%"=="D" Set [email protected]
If "%_DataLine:~1,1%"=="E" Set [email protected]
If "%_DataLine:~1,1%"=="X" Set [email protected]
Set _DataLine=%_Above%%_DataLine%
Set _Above=
For /F "Tokens=1-4 [email protected]" %%I In ("%_DataLine%") Do Call :_Split "%%I" "%%J" "%%K" "%%L"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Determines which field each parameter belongs in based on the first letter
:_Split
Set _P1=%~1
Set _P2=%~2
Set _P3=%~3
Set _P4=%~4
If NOT "%_P1%"=="" Call :%_P1:~0,1% "%_P1:~1%"
If NOT "%_P2%"=="" Call :%_P2:~0,1% "%_P2:~1%"
If NOT "%_P3%"=="" Call :%_P3:~0,1% "%_P3:~1%"
If NOT "%_P4%"=="" Call :%_P4:~0,1% "%_P4:~1%"
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Assign data to variables. Disc and Eff are Converted
:: to ranges with a minimum length in months of _DiscMinPeriod
:: and _EffMinPeriod. Ex, with a 3 month minimum, if Desc
:: date is Jan, it will become a range of Sept-Jan.
:A
Set _Above=%~1
Goto :EOF
:D
Set _Disc=%~1
Set _Disc=%_Disc:/=,%
Call :_Convert %_Disc%
Set _Disc=%_P1%
Set /A _P2=_P1/100-_DiscMinPeriod
If %_P2% LEQ 0 Set /A _P2=_P2+12
Set _Disc=%_P2%/1-%~1
Goto :EOF
:E
Set _Eff=%~1
Set _Eff=%_Eff:/=,%
Call :_Convert %_Eff%
Set _Eff=%_P1%
Set /A _P2=_P1/100+_EffMinPeriod
If %_P2% GEQ 12 Set /A _P2=_P2-12
Set _Eff=%~1-%_P2%/31
Set _Eff=%_Eff:,=%
Goto :EOF
:X
Set _Exc=%~1
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Exc and Disc are processed here. Each parameter except
:: Exc will be a single date range, Exc can be multiple
:: date ranges, or a single date. The - and / are removed
:: Then another call is made to separare the month and date.
:: These are converted to numbers for comparison.
:: Result is returned in the _Match variable
::          0 = SearchDate is not in the range
::          1 = SearchDate is in the range
:_Parse1
Set _Match=0
If "%1"=="" Goto :EOF
Set _Date=%1
:: Replace - and / with , so month and day will be two parameters
:: a range will be four parameters
Set _Date=%_Date:-=,%
Set _Date=%_Date:/=,%
Call :_Check %_Date%
If %_Match%==1 Goto :EOF
Shift
Goto _Parse1
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Above and Eff are processed here. Done separately so
:: EndDate can be checked
:_Parse2
Set _Match=0
If "%1"=="" Goto :EOF
Set _Date=%1
:: Replace - and / with , so month and day will be two parameters
:: a range will be four parameters
Set _Date=%_Date:-=,%
Set _Date=%_Date:/=,%
Call :_Check %_Date%
If %_Match%==1 Goto :EOF
If %_EndDate% GTR %_SearchDate% (
  If %_SearchDate% LSS %_P1% If %_EndDate% GEQ %_P1% Set _Match=1
) Else (
  If %_SearchDate% LSS %_P1% If %_P1% LEQ 1231  Set _Match=1
  If %_P1% GEQ 101 If %_EndDate% GEQ %_P1% Set _Match=1
)
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:_Check
:: Will be two or four parameters
Set /A _P1=%1*100+%2
:: If Date Range, calc end date, set to 0 if not range
If "%~3"=="" (Set _P2=0) Else (Set /A _P2=%3*100+%4)
:: if Range doesn't span end of year check if search date is within the range.
:: If Range spans end of year, check if date between start and 12/31 and 1/1 and end
If "%~3"=="" (
:: Single Date
  If %_P1%==%_SearchDate% Set _Match=1
) Else (
  If %_P2% GEQ %_P1% (
    If %_SearchDate% GEQ %_P1% If %_SearchDate% LEQ %_P2% Set _Match=1
  ) Else (
    If %_SearchDate% GEQ %_P1% If %_SearchDate% LEQ 1231 Set _Match=1
    If %_SearchDate% GEQ 101 If %_SearchDate% LEQ %_P2% Set _Match=1
  )
)
Goto :EOF
:::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Convert a date or date range to one or two numbers
:: Number= Month*100+day, so number is mmDD
:: Will be two or four parameters
:_Convert
Set /A _P1=%1*100+%2
Goto :EOF
```


----------



## fscldhppr (Mar 29, 2010)

Sorry for the late reply...

I changed the....

Set _ValidPeriod=1

to

Set _ValidPeriod=7

Is that correct? If so the batch file just disappears when hitting ENTER.

When I replace the value back to =1, it works as before.

I've attached the entire United Airlines date.txt.... so you can get an idea of run-time, etc.


----------



## TheOutcaste (Aug 8, 2007)

Arg, somehow an extra parenthesis snuck in on line 32:
If %_ValidPeriod% LSS 2 (Set _VP=1) Else Set _VP=2*)*
Remove that ending parenthesis.
Yes, set the *Set _ValidPeriod=* value to 7, didn't realize I'd left it at 1.

I've edited the above code to change those two items.
now to run it against your sample file.


----------



## fscldhppr (Mar 29, 2010)

OK, just to let you know....the file included in the .zip is the one I modified to not go beyond Dec. 31. For some reason I must not have saved the original...but will re-do the timetable in it's entirety and upload it shortly.


----------



## fscldhppr (Mar 29, 2010)

Here is the date.txt for UAL with all dates unchanged.

I actually had to use 2 zip files, as the forum only allows a max of 500kb upload for each one. Just combine the 2 for the complete date.txt.

I ran it through the new batch file and will report back the results. At first glance though it appears there may be one last glitch....that having to do with flights that only fly specific days of the week, and (unfortunately) that variable may have to be taken into account also when deciding if those flight segments stay or go.

For reference the UAL pdf can be found here...

http://www.uatimetable.com/United.pdf?linkTitle=Worldwide+timetable+PDF

The ndate.txt output file included several duplicate flights.....which look like this(info here directly from the pdf itself....not the date.txt).

1500 1812 UA686 319 X6 0 
Above Eff. 12/19, Exc. 12/20 - 1/3

1503 1809 UA686 319 X7 0 
Above 12/16 - 12/23

1503 1818 UA686 319 X6 0 
Above Disc. 12/15

The flight segment in blue(Above Disc. 12/15)was successfully removed by the batch file....the other 2 remained as valid flights. That makes sense as they are both within the weekly time period of 12/17-12/23. However each flight(which is actually the same flight) also includes X(exclude this day)days...the first flight X6(excludes Saturday), the second X7(excludes Sunday).

So using the Mr. Spock method(heh)...."if you eliminate the _impossible_, whatever remains, no matter how improbable, is possible"

What United is saying(I think)is that the first flight starts on the 19th(Sunday)but excludes not only 12/20-1/3(taking out 12/20-12/23 from our weekly time period), but also all Saturday flights within that range, leaving only the Sunday flight(12/19)standing.

The second flight picks up the slack for the week as it includes all flights within our weekly time period(12/17-12/23)but leaves out the Sunday flight(X7)....picked up by the first segment.

Why United decides a 3 minute dep./arr. time difference requires a completely different flight segment is beyond me.

Anyway, I am going to take a closer look at the 2 output files to look for duplicate flights and such and see what other little surprises our friends at UAL decided to throw in their timetable ....and get back to you.


----------



## fscldhppr (Mar 29, 2010)

During the holiday's I had a "little" time off to check the output file in more depth and the concluded that, while adding further variables to the batch file to take into account any/all individual date patterns might help eliminate a decent percentage of errors to the final product....I really do not see a viable method of being able to remove pattern flaws that are just built in the structure of the(pdf)timetable data itself, not to mention the just plain old fact that, other data errors, in which the timetable data has mistakenly added or subtracted flights which are/not valid for the time period in question, still make up probably 30-40% of total errors on the final product.

Those have to be manually edited out anyway....and unfortunately(work/family)time constraints being what they are currently do not allow me much time to dedicate to, what is basically, a hobby(flight simulator)of mine and others....I'm sure your limited time also can be better utilized on more important(work/school)related projects you/others may have.

Bottom line is, *many thanks to TheOutcaste for all your work on this project*....I'll have to study up on batch/script file language/structures...perhaps if I have any questions I can come back and use your knowledge again on future projects.

Thanks again, and Happy New Year guys.


----------



## Coolwillys (Jan 15, 2011)

Hi guys and sorry to jump in the middle of your conversation.... If it's possible I will be very grateful if you could provide a little more information about this subject. The thing is that I also need a script to extract the data from the Star Alliance timetable pdf, but I really don't have a clue on how to do it. A little while ago, browsing the Internet, I also found that link you mentioned before (http://nothing.tmtm.com/2009/05/more-bmi-hacking/) where the guy parsed all the data out of S.A. PDF, but it's as old as 05/2009 and I don't know how to update it. I'm pretty good with computers & Windows, but a total newbie when it comes down to script & programming. I do learn fast, but that's about it !! As a matter of fact, my first stupid question would be: How do you run the script that you guys have posted in order to obtain the output file ??
Anyway, for me, the best would be a .CSV output, exactly like the guy in the link did it. I tried several times to get in touch with him but never received a response. Maybe you guys are willing to give me some clues or at least a starting point. Thanks in advance for all your time.

Sincerely

Will


----------

