use djangoCMS as backbone and add below features to it
A. scraper module as an app
1. support multi-threading and dynamic ip
2. support scrap by company number range e.g:
3. database: MySQL
schema table:
-Company:Company:Name,CompanyNumber,NZBN,CompanyStatus,EntityType,IncorporationDate,ARFilingMonth,ExtensiveShareholding,UltimateHoldingCompanyNumber,
-AdditionalCompanyInformation:Trading Name, NZSXCode,Email, BIC(BusinessClassificationCode)
-Address: CompanyNumber,Type(RegisteredOfficeAddress,AddressService,ResidentialAddress(Director))
-Director: FullLegalName, ResidentialAddress, AppointmentDate, CeasedDate(for historic directors, ref:[login to view URL])
-ShareHoldings: TotalShares, ExtensiveShareholding,
-ShareAllocation: Shares, CompanyNumber, PersonName, IsDirector, Address
-HistoricShareholders: Name,VacationDate
-*Ultimate holding company (ref:[login to view URL])
-*BIC ref([login to view URL])
-*ShareAllocation mixed with company and person: ref [login to view URL]
-*All Dates should be date data type not string
4. Data access layer:
SQLAlchemy
B. WebApi(Json)
-token(64bit)
-ObjectInfo[{CompanyName:ABC limited,Address:[Type:'', value:'']}, Director:[{Name:abc,AppointmentDate:},{Name:bcd}],ShareAllocation:[{},{}]]
-scurity: SSL/https
C. admin pages to manage scrapers
1. a page to list of all scrapers
2. a detail page to display scraper defined in A.
3. in page#C2, text box to put in a specific number to scrap a company or a range to scrap a range of companies
D. front end pages to display
1. search box to search company by NZBN/CompanyNumber or search director name
2. page to display result in #D1
*** UI(Angular4 + Bootstrap4)
IMPORTANT NOTE: Total price of this job is as high as you bid, no-negotiable.
Hi there,
We have done many projects in Python and related framework so can do this in Scrapy or other framework. If interested please come to the chat to discuss more.
Thanks
Hello!
I have deep experience with python and django framework.
If you want to check, you can find 'nightagenda' work in my portfolio.
So, good luck in finding necessary specialist.