Jul 24 15:38:51.383961 ip-10-0-133-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 15:38:51.383973 ip-10-0-133-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 15:38:51.383983 ip-10-0-133-134 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 15:38:51.384810 ip-10-0-133-134 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 15:39:01.471937 ip-10-0-133-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 15:39:01.471961 ip-10-0-133-134 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3c978303a83445aeb4aa5ae01a02da8f --
Jul 24 15:40:58.342452 ip-10-0-133-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 15:40:58.693638 ip-10-0-133-134 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:40:58.693638 ip-10-0-133-134 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 15:40:58.693638 ip-10-0-133-134 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:40:58.693638 ip-10-0-133-134 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 15:40:58.693638 ip-10-0-133-134 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:40:58.694505 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.694364    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 15:40:58.699602 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699580    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.699602 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699599    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.699602 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699604    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.699602 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699609    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699613    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699618    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699623    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699627    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699631    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699635    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699638    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699643    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699647    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699651    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699654    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699658    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699663    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699668    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699674    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699678    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699682    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699687    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.699867 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699692    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699698    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699704    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699708    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699712    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699717    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699721    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699726    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699730    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699734    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699738    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699742    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699745    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699750    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699754    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699759    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699765    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699769    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699773    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.700660 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699777    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699781    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699786    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699793    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699799    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699804    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699808    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699812    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699817    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699821    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699825    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699829    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699834    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699838    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699842    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699846    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699850    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699855    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699859    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699863    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.701521 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699867    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699870    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699875    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699879    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699883    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699887    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699892    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699896    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699901    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699905    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699911    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699915    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699919    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699923    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699927    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699932    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699936    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699940    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699945    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.702293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699949    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699953    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699957    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699961    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699965    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699969    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699973    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699978    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699982    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699988    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699993    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.699998    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700002    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700006    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700010    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700014    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700019    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700023    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.700028    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701220    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.702750 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701231    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701236    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701241    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701245    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701250    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701254    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701258    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701264    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701270    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701275    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701279    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701284    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701288    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701292    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701296    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701300    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701304    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701308    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701313    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.703569 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701317    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701321    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701325    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701329    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701333    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701337    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701341    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701345    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701349    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701353    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701358    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701364    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701371    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701376    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701381    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701386    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701390    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701394    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701398    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701403    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.704425 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701407    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701411    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701415    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701420    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701424    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701428    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701435    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701440    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701445    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701449    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701453    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701458    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701462    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701466    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701470    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701474    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701479    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701483    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701487    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.705254 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701492    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701496    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701500    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701504    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701508    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701512    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701517    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701520    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701524    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701528    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701532    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701536    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701542    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701546    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701550    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701555    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701559    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701564    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701569    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701573    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.705741 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701577    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701581    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701585    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701589    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701594    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701598    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701602    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701606    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701612    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701616    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701620    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701624    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701629    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701635    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701641    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701646    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701650    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701654    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701658    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.701663    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.706360 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703056    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703067    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703072    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703077    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703083    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703089    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703094    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703099    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703103    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703107    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703111    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703116    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703120    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703144    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703148    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703152    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703156    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703161    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703164    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703169    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.707057 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703173    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703177    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703181    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703187    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703191    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703196    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703200    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703204    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703208    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703212    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703216    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703220    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703225    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703229    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703233    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703238    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703243    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703248    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703252    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703256    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.707620 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703261    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703265    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703269    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703273    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703278    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703282    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703286    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703290    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703294    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703299    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703303    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703307    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703313    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703319    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703323    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703328    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703332    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703336    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703340    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.708102 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703344    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703351    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703355    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703359    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703363    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703367    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703374    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703380    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703386    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703390    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703395    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703400    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703404    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703408    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703412    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703416    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703419    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703424    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703428    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.708671 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703432    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703436    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703440    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703444    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703448    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703452    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703455    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703459    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703464    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703468    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703472    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703479    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703483    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703488    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703494    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703498    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703503    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703508    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703513    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.709154 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703517    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.703522    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703637    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703649    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703659    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703666    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703673    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703679    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703685    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703692    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703697    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703702    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703708    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703713    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703718    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703723    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703727    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703732    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703736    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703741    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703748    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703752    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703757    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703762    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 15:40:58.709634 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703768    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703774    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703778    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703786    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703792    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703796    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703807    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703812    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703817    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703821    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703828    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703832    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703837    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703841    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703846    2567 flags.go:64] FLAG: --enable-server="true"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703852    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703860    2567 flags.go:64] FLAG: --event-burst="100"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703865    2567 flags.go:64] FLAG: --event-qps="50"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703869    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703874    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703879    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703886    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703890    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703895    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703900    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 24 15:40:58.710243 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703904    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703909    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703913    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703917    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703922    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703927    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703931    2567 flags.go:64] FLAG: --feature-gates=""
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703937    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703942    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703946    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703953    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703958    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703963    2567 flags.go:64] FLAG: --help="false"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703967    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-133-134.ec2.internal"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703973    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703977    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703982    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703987    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703993    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.703997    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704001    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704006    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704010    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 15:40:58.710836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704019    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704024    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704029    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704033    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704038    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704043    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704048    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704052    2567 flags.go:64] FLAG: --lock-file=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704056    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704061    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704066    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704075    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704079    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704084    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704088    2567 flags.go:64] FLAG: --logging-format="text"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704093    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704097    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704102    2567 flags.go:64] FLAG: --manifest-url=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704107    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704115    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704120    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704145    2567 flags.go:64] FLAG: --max-pods="110"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704150    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704155    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704159    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 15:40:58.711478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704163    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704168    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704172    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704177    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704189    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704193    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704198    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704204    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704210    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704220    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704225    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704230    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704234    2567 flags.go:64] FLAG: --port="10250"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704239    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704244    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0424e58f3af5d8faf"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704249    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704253    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704258    2567 flags.go:64] FLAG: --register-node="true"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704262    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704268    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704273    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704277    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704282    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704287    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704292    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 15:40:58.712141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704297    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704304    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704309    2567 flags.go:64] FLAG: --runonce="false"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704314    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704319    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704324    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704328    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704333    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704338    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704342    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704347    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704353    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704358    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704362    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704367    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704374    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704380    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704384    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704393    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704398    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704403    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704410    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704414    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704419    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704423    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 15:40:58.712844 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704428    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704432    2567 flags.go:64] FLAG: --v="2"
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704438    2567 flags.go:64] FLAG: --version="false"
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704444    2567 flags.go:64] FLAG: --vmodule=""
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704450    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.704455    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704665    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704672    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704677    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704682    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704686    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704690    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704696    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704700    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704704    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704708    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704713    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704717    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704721    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704725    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704729    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.713473 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704733    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704739    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704743    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704748    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704752    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704756    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704761    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704765    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704769    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704773    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704777    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704784    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704789    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704794    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704799    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704803    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704807    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704813    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704819    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.714034 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704824    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704828    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704833    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704837    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704841    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704845    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704850    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704853    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704858    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704862    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704866    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704871    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704875    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704879    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704887    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704893    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704897    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704902    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704906    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.714817 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704910    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704914    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704919    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704923    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704927    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704931    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704935    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704940    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704944    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704949    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704953    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704957    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704962    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704966    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704970    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704974    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704978    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704982    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704986    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704990    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.715406 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704994    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.704998    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705002    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705006    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705010    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705014    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705018    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705024    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705028    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705032    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705036    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705041    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705045    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705049    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705053    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705058    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705062    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705066    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705070    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705074    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.715926 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705078    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705083    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705087    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705091    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705095    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.705099    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.705817    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713112    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713143    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713158    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713164    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713218    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713224    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713228    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713231    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.716443 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713234    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713237    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713240    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713242    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713245    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713247    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713250    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713252    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713256    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713260    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713263    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713265    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713268    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713271    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713274    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713277    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713280    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713282    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713285    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.716810 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713287    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713290    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713292    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713295    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713297    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713300    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713304    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713307    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713309    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713312    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713314    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713316    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713319    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713321    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713324    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713326    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713329    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713331    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713334    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713336    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.717293 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713339    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713341    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713343    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713346    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713348    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713350    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713353    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713355    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713358    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713360    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713364    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713367    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713370    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713373    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713376    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713378    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713381    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713383    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713386    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.717780 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713389    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713391    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713394    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713396    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713399    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713401    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713403    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713406    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713408    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713411    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713413    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713416    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713418    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713420    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713423    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713425    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713428    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713430    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713433    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713435    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.718405 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713437    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713440    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713442    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713445    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713447    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713450    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713452    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713455    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713457    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713460    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713462    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713465    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713469    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713473    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713476    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713479    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.718895 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713482    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713488    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713603    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713607    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713611    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713614    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713617    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713620    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713622    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713625    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713628    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713631    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713633    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713636    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713640    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:40:58.719333 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713642    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713645    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713648    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713650    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713652    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713655    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713657    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713660    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713662    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713665    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713667    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713670    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713673    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713676    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713678    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713681    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713683    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713686    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713688    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713691    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:40:58.719697 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713693    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713695    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713698    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713700    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713702    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713705    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713707    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713709    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713712    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713714    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713717    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713719    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713722    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713724    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713727    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713729    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713732    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713734    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713737    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713740    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:40:58.720220 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713742    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713745    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713747    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713750    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713752    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713755    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713757    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713760    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713762    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713765    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713768    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713770    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713772    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713775    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713777    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713780    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713783    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713785    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713789    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713792    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:40:58.720704 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713795    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713798    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713801    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713803    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713806    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713808    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713810    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713813    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713815    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713817    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713820    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713822    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713825    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713827    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713829    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713832    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713834    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713838    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713841    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:40:58.721195 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713843    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713846    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713849    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713852    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713855    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713859    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:58.713862    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.713866    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.714404    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.717776    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.719091    2567 server.go:1019] "Starting client certificate rotation"
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.719160    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.719246    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:40:58.721653 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.721538    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:40:58.722861 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.722846    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:40:58.731887 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.731868    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 24 15:40:58.734321 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.734305    2567 log.go:25] "Validated CRI v1 image API"
Jul 24 15:40:58.735206 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.735180    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 15:40:58.739686 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.739666    2567 fs.go:135] Filesystem UUIDs: map[314dd0a7-8f97-475a-8c9b-e1f664d95ea5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bae6e8ad-a65e-48f9-b564-06c1de1e62b5:/dev/nvme0n1p3]
Jul 24 15:40:58.739759 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.739687    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 15:40:58.744693 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.744586    2567 manager.go:216] Machine: {Timestamp:2026-07-24 15:40:58.743343527 +0000 UTC m=+0.295583291 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3166360 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c8b59fa990e4e71a3bc555395c3df SystemUUID:ec2c8b59-fa99-0e4e-71a3-bc555395c3df BootID:3c978303-a834-45ae-b4aa-5ae01a02da8f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:61:49:21:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:61:49:21:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:70:85:4c:9f:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 15:40:58.744693 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.744688    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 15:40:58.744796 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.744772    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 15:40:58.747427 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.747396    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 15:40:58.747572 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.747429    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 15:40:58.748080 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.748070    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 15:40:58.748113 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.748082    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 15:40:58.748113 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.748090    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:40:58.748676 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.748665    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:40:58.750238 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.750227    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 15:40:58.754204 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754187    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:40:58.754303 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754290    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:40:58.754471 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754462    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 15:40:58.754500 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754493    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 15:40:58.754530 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754503    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 15:40:58.754530 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754514    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 15:40:58.754530 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754524    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 24 15:40:58.754611 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.754534    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 15:40:58.755433 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.755420    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:40:58.755493 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.755444    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:40:58.758011 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.757878    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 15:40:58.759460 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.759442    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 15:40:58.759547 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.759475    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 15:40:58.761233 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761221    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761240    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761251    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761260    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761274    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761280    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761289    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 15:40:58.761293 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761294    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 15:40:58.761473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761301    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 15:40:58.761473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761307    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 15:40:58.761473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761317    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 15:40:58.761473 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.761326    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 15:40:58.762024 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.762015    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 15:40:58.762024 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.762024    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 15:40:58.765545 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.765529    2567 server.go:1268] "Started kubelet"
Jul 24 15:40:58.765644 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.765621    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 15:40:58.765747 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.765693    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 15:40:58.765849 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.765801    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 15:40:58.766540 ip-10-0-133-134 systemd[1]: Started Kubernetes Kubelet.
Jul 24 15:40:58.767280 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.767182    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 15:40:58.767603 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.767584    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 15:40:58.769373 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.769348    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 15:40:58.769481 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.769401    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 15:40:58.769481 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.769466    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 15:40:58.772598 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.772576    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 15:40:58.775040 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.775020    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 15:40:58.775040 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.775035    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 15:40:58.775216 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.775093    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:58.775690 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.775659    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 15:40:58.775803 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.775665    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 15:40:58.775891 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.775874    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 15:40:58.776014 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.774911    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-134.ec2.internal.18c5440f0d9775e9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-134.ec2.internal,UID:ip-10-0-133-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-134.ec2.internal,},FirstTimestamp:2026-07-24 15:40:58.765497833 +0000 UTC m=+0.317737598,LastTimestamp:2026-07-24 15:40:58.765497833 +0000 UTC m=+0.317737598,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-134.ec2.internal,}"
Jul 24 15:40:58.776105 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776021    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 15:40:58.776105 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776030    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 15:40:58.776105 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776063    2567 factory.go:153] Registering CRI-O factory
Jul 24 15:40:58.776105 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776075    2567 factory.go:223] Registration of the crio container factory successfully
Jul 24 15:40:58.776292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776140    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 15:40:58.776292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776152    2567 factory.go:55] Registering systemd factory
Jul 24 15:40:58.776292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776158    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 24 15:40:58.776292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776178    2567 factory.go:103] Registering Raw factory
Jul 24 15:40:58.776292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776193    2567 manager.go:1196] Started watching for new ooms in manager
Jul 24 15:40:58.776755 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.776740    2567 manager.go:349] Starting recovery of all containers
Jul 24 15:40:58.787457 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.787436    2567 manager.go:354] Recovery completed
Jul 24 15:40:58.787704 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.787474    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 15:40:58.787704 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.787571    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 15:40:58.789623 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.789604    2567 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 24 15:40:58.792778 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.792760    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:58.794524 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.794506    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:58.794583 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.794538    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:58.794583 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.794549    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:58.795076 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795063    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 15:40:58.795076 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795075    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 15:40:58.795178 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795090    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:40:58.795178 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795119    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:58.795759 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795746    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:58.795804 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795770    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:58.795804 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.795781    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:58.797829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.797817    2567 policy_none.go:49] "None policy: Start"
Jul 24 15:40:58.797867 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.797833    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 15:40:58.797867 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.797844    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 15:40:58.799198 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.799104    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-134.ec2.internal.18c5440f0f525e83  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-134.ec2.internal,UID:ip-10-0-133-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-134.ec2.internal,},FirstTimestamp:2026-07-24 15:40:58.794524291 +0000 UTC m=+0.346764055,LastTimestamp:2026-07-24 15:40:58.794524291 +0000 UTC m=+0.346764055,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-134.ec2.internal,}"
Jul 24 15:40:58.799826 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.799814    2567 policy_none.go:47] "Start"
Jul 24 15:40:58.813809 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.813611    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-134.ec2.internal.18c5440f0f52ab02  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-134.ec2.internal,UID:ip-10-0-133-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-134.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-134.ec2.internal,},FirstTimestamp:2026-07-24 15:40:58.794543874 +0000 UTC m=+0.346783638,LastTimestamp:2026-07-24 15:40:58.794543874 +0000 UTC m=+0.346783638,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-134.ec2.internal,}"
Jul 24 15:40:58.814326 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.814310    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-clw4x"
Jul 24 15:40:58.822752 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.822734    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-clw4x"
Jul 24 15:40:58.824270 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.824187    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-134.ec2.internal.18c5440f0f52d0b3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-134.ec2.internal,UID:ip-10-0-133-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-134.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-134.ec2.internal,},FirstTimestamp:2026-07-24 15:40:58.794553523 +0000 UTC m=+0.346793287,LastTimestamp:2026-07-24 15:40:58.794553523 +0000 UTC m=+0.346793287,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-134.ec2.internal,}"
Jul 24 15:40:58.830533 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.830517    2567 manager.go:337] "Starting Device Plugin manager"
Jul 24 15:40:58.830621 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.830553    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 15:40:58.830621 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.830567    2567 server.go:87] "Starting device plugin registration server"
Jul 24 15:40:58.830819 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.830806    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 15:40:58.830878 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.830821    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 15:40:58.830968 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.830952    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 15:40:58.831055 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.831039    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 15:40:58.831055 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.831052    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 15:40:58.831540 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.831522    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 15:40:58.831622 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.831569    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:58.931693 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.931660    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:58.933911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.932904    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:58.933911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.932943    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:58.933911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.932955    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:58.933911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.932982    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-134.ec2.internal"
Jul 24 15:40:58.941226 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.941199    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 15:40:58.942621 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.942604    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 15:40:58.942710 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.942649    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 15:40:58.942710 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.942675    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 15:40:58.942800 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.942714    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 15:40:58.947605 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.947553    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:40:58.948536 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:58.948521    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-134.ec2.internal"
Jul 24 15:40:58.948594 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:58.948545    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-134.ec2.internal\": node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.017428 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.017397    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.043628 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.043590    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"]
Jul 24 15:40:59.043712 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.043701    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:59.045440 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.045419    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:59.045557 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.045451    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:59.045557 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.045460    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:59.047859 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.047846    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:59.047972 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.047955    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.048023 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.047989    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:59.048583 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048567    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:59.048640 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048595    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:59.048640 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048605    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:59.048640 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048567    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:59.048727 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048660    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:59.048727 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.048675    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:59.050919 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.050906    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.050964 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.050930    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:40:59.051600 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.051584    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:40:59.051671 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.051611    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:40:59.051671 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.051624    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:40:59.072859 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.072840    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-134.ec2.internal\" not found" node="ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.077041 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.077022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.077155 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.077056    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-134.ec2.internal\" not found" node="ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.077155 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.077055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.077155 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.077149    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4fd32fde91ac02b5f7cfbf50c6d4bcc8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-134.ec2.internal\" (UID: \"4fd32fde91ac02b5f7cfbf50c6d4bcc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.118297 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.118249    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.177911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4fd32fde91ac02b5f7cfbf50c6d4bcc8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-134.ec2.internal\" (UID: \"4fd32fde91ac02b5f7cfbf50c6d4bcc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.178001 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.178001 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.178001 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177978    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4fd32fde91ac02b5f7cfbf50c6d4bcc8-config\") pod \"kube-apiserver-proxy-ip-10-0-133-134.ec2.internal\" (UID: \"4fd32fde91ac02b5f7cfbf50c6d4bcc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.178001 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.178144 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.177977    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d0a1fc27b2b71d2f6d0b5866eb66fcf-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal\" (UID: \"2d0a1fc27b2b71d2f6d0b5866eb66fcf\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.219435 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.219376    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.320211 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.320173    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.375400 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.375373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.380518 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.380488    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.420576 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.420549    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.521152 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.521111    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.621644 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.621604    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-134.ec2.internal\" not found"
Jul 24 15:40:59.708942 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.708912    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:40:59.718113 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.718084    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 15:40:59.718286 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.718265    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:40:59.718324 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.718265    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:40:59.755345 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.755322    2567 apiserver.go:52] "Watching apiserver"
Jul 24 15:40:59.776073 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.776051    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 15:40:59.776258 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.776057    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.776258 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.776202    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 15:40:59.776666 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.776643    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-59zdt","kube-system/konnectivity-agent-7wxwm","openshift-cluster-node-tuning-operator/tuned-hg746","openshift-dns/node-resolver-kt2bk","openshift-image-registry/node-ca-wkglg","openshift-multus/multus-4nkwr","openshift-multus/network-metrics-daemon-jbfbj","openshift-network-operator/iptables-alerter-mzlh8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr","openshift-multus/multus-additional-cni-plugins-69fgt","openshift-network-diagnostics/network-check-target-44n2k","openshift-ovn-kubernetes/ovnkube-node-89rdl"]
Jul 24 15:40:59.780309 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.780281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.780402 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.780356    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:40:59.782408 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.782377    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.784597 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.784580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.784686 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.784676    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 15:40:59.784744 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.784683    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 15:40:59.784801 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.784687    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-59nmh\""
Jul 24 15:40:59.786784 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.786768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.788831 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.788811    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.789800 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.789779    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.789800 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.789791    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.789972 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.789854    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7mk27\""
Jul 24 15:40:59.790492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.790238    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:40:59.790492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.790327    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.790833    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.790852    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5qw2l\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.791015    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.791368    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.791369    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.791543    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-576wp\""
Jul 24 15:40:59.791774 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.791641    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.792641 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.792604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.793438 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.793421    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:40:59.794957 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.794936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:40:59.795049 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.795017    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:40:59.797559 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797010    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 15:40:59.797559 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797033    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.797559 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797093    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 15:40:59.797559 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797271    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.797835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.797835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.797661    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xxnrf\""
Jul 24 15:40:59.799818 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.799802    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 15:40:59.799922 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.799904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.799991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.799904    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hj2m4\""
Jul 24 15:40:59.800470 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.800447    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.800547 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.800485    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.802222 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.802208    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.802449 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.802416    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 15:40:59.802662 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.802647    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.802731 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.802652    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-47phx\""
Jul 24 15:40:59.803103 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.803087    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.803724 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.803709    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:40:59.804506 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.804489    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 15:40:59.804681 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.804666    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:40:59.804772 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.804752    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:40:59.804949 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.804932    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 15:40:59.805286 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.805272    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-knbwm\""
Jul 24 15:40:59.807084 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.807065    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal"]
Jul 24 15:40:59.807193 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.807089    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal"]
Jul 24 15:40:59.807193 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.807180    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.809720 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.809699    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 15:40:59.809887 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.809870    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 15:40:59.810829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.810809    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9qw9q\""
Jul 24 15:40:59.810926 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.810841    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 15:40:59.811014 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.810999    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 15:40:59.811093 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.811074    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 15:40:59.811166 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.811090    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 15:40:59.819974 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.819953    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfnpr"
Jul 24 15:40:59.829246 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.829210    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:35:58 +0000 UTC" deadline="2028-04-30 05:08:53.070765502 +0000 UTC"
Jul 24 15:40:59.829246 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.829244    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15493h27m53.241524326s"
Jul 24 15:40:59.837776 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.837752    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfnpr"
Jul 24 15:40:59.876941 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.876923    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 15:40:59.882318 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882297    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.882392 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53a3261e-72d6-4de2-a75a-0256b5ee1513-konnectivity-ca\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.882392 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:40:59.882392 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-systemd-units\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.882492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-lib-modules\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-os-release\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.882492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-var-lib-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.882492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-script-lib\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.882492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-sys\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882502    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-serviceca\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-binary-copy\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-bin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-etc-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-netd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.882652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-kubernetes\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-conf\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882671    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-tuned\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-tmp\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882737    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445e9c17-f645-469d-b0a1-1807c3e9a08e-hosts-file\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-hostroot\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882792    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-cnibin\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-os-release\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.882820 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-kubelet\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hf59\" (UniqueName: \"kubernetes.io/projected/d8bfae1f-8145-47cd-8531-da65a17409e3-kube-api-access-6hf59\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-host\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53a3261e-72d6-4de2-a75a-0256b5ee1513-agent-certs\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-multus\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-conf-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882976    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-kubelet\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.882998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-slash\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883018    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-run\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-daemon-config\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883085    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysconfig\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-systemd\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-var-lib-kubelet\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883164    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bg7m\" (UniqueName: \"kubernetes.io/projected/785537e2-76bf-4238-9c65-cbb4ff59cf37-kube-api-access-7bg7m\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cnibin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883225    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cni-binary-copy\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-config\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-env-overrides\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5q54\" (UniqueName: \"kubernetes.io/projected/f517fb98-037e-45ae-99c0-a68b8304ef0a-kube-api-access-d5q54\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-k8s-cni-cncf-io\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-ovn\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883329    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883362    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr7mg\" (UniqueName: \"kubernetes.io/projected/f7afd27a-11a7-4701-9b08-408dcc75cc08-kube-api-access-hr7mg\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:40:59.883413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d44c172d-a3c0-4cf0-998e-4056f3fde862-host-slash\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-system-cni-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-etc-kubernetes\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883453    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b8wt\" (UniqueName: \"kubernetes.io/projected/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-kube-api-access-8b8wt\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883474    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d44c172d-a3c0-4cf0-998e-4056f3fde862-iptables-alerter-script\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883514    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-device-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-sys-fs\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883636    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-systemd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883649    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-dbus\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883662    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-host\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2nf4\" (UniqueName: \"kubernetes.io/projected/d44c172d-a3c0-4cf0-998e-4056f3fde862-kube-api-access-l2nf4\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-netns\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv4v9\" (UniqueName: \"kubernetes.io/projected/42c1cc67-5c1b-46b3-b120-1d89c93e832e-kube-api-access-jv4v9\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.883909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883743    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-log-socket\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-bin\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8bfae1f-8145-47cd-8531-da65a17409e3-ovn-node-metrics-cert\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-modprobe-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883809    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883830    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-registration-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883854    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-socket-dir-parent\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-node-log\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-kubelet-config\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xsqj\" (UniqueName: \"kubernetes.io/projected/37801522-98d0-4c13-a06a-2a700a32d141-kube-api-access-8xsqj\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.883993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-netns\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.884016    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445e9c17-f645-469d-b0a1-1807c3e9a08e-tmp-dir\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.884043    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxglt\" (UniqueName: \"kubernetes.io/projected/445e9c17-f645-469d-b0a1-1807c3e9a08e-kube-api-access-bxglt\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.884057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-system-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.884086    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-multus-certs\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.884393 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.884108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-socket-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.917011 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:59.916963    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0a1fc27b2b71d2f6d0b5866eb66fcf.slice/crio-0df6820efbf65ecf95a3afb2f1c536f693436563a7b37ecb9f8057f742d44810 WatchSource:0}: Error finding container 0df6820efbf65ecf95a3afb2f1c536f693436563a7b37ecb9f8057f742d44810: Status 404 returned error can't find the container with id 0df6820efbf65ecf95a3afb2f1c536f693436563a7b37ecb9f8057f742d44810
Jul 24 15:40:59.918493 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:40:59.918471    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd32fde91ac02b5f7cfbf50c6d4bcc8.slice/crio-eafd9982deae8177a7f585fe67f452dafe952af07fe66a2e3d8debbcef670152 WatchSource:0}: Error finding container eafd9982deae8177a7f585fe67f452dafe952af07fe66a2e3d8debbcef670152: Status 404 returned error can't find the container with id eafd9982deae8177a7f585fe67f452dafe952af07fe66a2e3d8debbcef670152
Jul 24 15:40:59.922950 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.922931    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:40:59.936867 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.936844    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:40:59.946062 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.945996    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal" event={"ID":"4fd32fde91ac02b5f7cfbf50c6d4bcc8","Type":"ContainerStarted","Data":"eafd9982deae8177a7f585fe67f452dafe952af07fe66a2e3d8debbcef670152"}
Jul 24 15:40:59.946934 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.946907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal" event={"ID":"2d0a1fc27b2b71d2f6d0b5866eb66fcf","Type":"ContainerStarted","Data":"0df6820efbf65ecf95a3afb2f1c536f693436563a7b37ecb9f8057f742d44810"}
Jul 24 15:40:59.984985 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.984953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.985079 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.984991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-registration-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.985079 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-socket-dir-parent\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985079 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-node-log\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985093    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985096    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-registration-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.985237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-kubelet-config\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.985237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985183    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-socket-dir-parent\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xsqj\" (UniqueName: \"kubernetes.io/projected/37801522-98d0-4c13-a06a-2a700a32d141-kube-api-access-8xsqj\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-netns\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985315    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-kubelet-config\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445e9c17-f645-469d-b0a1-1807c3e9a08e-tmp-dir\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985333    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985357    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-run-netns\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985389    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxglt\" (UniqueName: \"kubernetes.io/projected/445e9c17-f645-469d-b0a1-1807c3e9a08e-kube-api-access-bxglt\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-system-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-multus-certs\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985498    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-system-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-socket-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985558    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-multus-certs\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985576    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53a3261e-72d6-4de2-a75a-0256b5ee1513-konnectivity-ca\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985696    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-socket-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985720    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/445e9c17-f645-469d-b0a1-1807c3e9a08e-tmp-dir\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-systemd-units\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-lib-modules\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-os-release\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985838    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-var-lib-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-script-lib\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-sys\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985891    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-os-release\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-serviceca\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.985936 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-binary-copy\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985937    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-lib-modules\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985964    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-bin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985993    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-etc-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-var-lib-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986042    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-netd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986061    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-sys\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-bin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-kubernetes\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-conf\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986169    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-kubernetes\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986193    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-tuned\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986216    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-etc-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-tmp\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986254    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-netd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986268    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445e9c17-f645-469d-b0a1-1807c3e9a08e-hosts-file\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-systemd-units\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.986717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986359    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-hostroot\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-cnibin\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-os-release\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986427    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/53a3261e-72d6-4de2-a75a-0256b5ee1513-konnectivity-ca\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.985289    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-node-log\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-kubelet\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hf59\" (UniqueName: \"kubernetes.io/projected/d8bfae1f-8145-47cd-8531-da65a17409e3-kube-api-access-6hf59\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-os-release\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986592    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-kubelet\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-host\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-serviceca\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53a3261e-72d6-4de2-a75a-0256b5ee1513-agent-certs\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986731    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-multus\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.987499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-conf-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-kubelet\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986803    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-etc-selinux\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-slash\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-run\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986854    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-host\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-script-lib\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-daemon-config\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.986911    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysconfig\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987105    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-slash\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-tuning-conf-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987053    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-cni-binary-copy\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-conf-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-systemd\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-var-lib-kubelet\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987885    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bg7m\" (UniqueName: \"kubernetes.io/projected/785537e2-76bf-4238-9c65-cbb4ff59cf37-kube-api-access-7bg7m\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.988593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysctl-conf\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988024    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-cnibin\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/445e9c17-f645-469d-b0a1-1807c3e9a08e-hosts-file\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-var-lib-cni-multus\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-systemd\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988175    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-sysconfig\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988229    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-var-lib-kubelet\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.988305    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.988482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:00.488438393 +0000 UTC m=+2.040678161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.988580    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.988736    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:00.488718076 +0000 UTC m=+2.040957827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988734    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-hostroot\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988784    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-run\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.987915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-kubelet\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988866    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cnibin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cni-binary-copy\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-config\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.988970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-env-overrides\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5q54\" (UniqueName: \"kubernetes.io/projected/f517fb98-037e-45ae-99c0-a68b8304ef0a-kube-api-access-d5q54\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-k8s-cni-cncf-io\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-ovn\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr7mg\" (UniqueName: \"kubernetes.io/projected/f7afd27a-11a7-4701-9b08-408dcc75cc08-kube-api-access-hr7mg\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d44c172d-a3c0-4cf0-998e-4056f3fde862-host-slash\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989223    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-system-cni-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-etc-kubernetes\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989309    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-daemon-config\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b8wt\" (UniqueName: \"kubernetes.io/projected/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-kube-api-access-8b8wt\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989642    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-ovn\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989717    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989792    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-env-overrides\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.989968    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d44c172d-a3c0-4cf0-998e-4056f3fde862-host-slash\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37801522-98d0-4c13-a06a-2a700a32d141-system-cni-dir\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.990170 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990105    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-etc-kubernetes\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cnibin\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990218    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-tmp\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d44c172d-a3c0-4cf0-998e-4056f3fde862-iptables-alerter-script\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990292    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-device-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990373    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-sys-fs\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-systemd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-dbus\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990519    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-host\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8bfae1f-8145-47cd-8531-da65a17409e3-ovnkube-config\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/53a3261e-72d6-4de2-a75a-0256b5ee1513-agent-certs\") pod \"konnectivity-agent-7wxwm\" (UID: \"53a3261e-72d6-4de2-a75a-0256b5ee1513\") " pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990551    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2nf4\" (UniqueName: \"kubernetes.io/projected/d44c172d-a3c0-4cf0-998e-4056f3fde862-kube-api-access-l2nf4\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-netns\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990610    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv4v9\" (UniqueName: \"kubernetes.io/projected/42c1cc67-5c1b-46b3-b120-1d89c93e832e-kube-api-access-jv4v9\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990619    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-k8s-cni-cncf-io\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-log-socket\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990678    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-systemd\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-log-socket\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-bin\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8bfae1f-8145-47cd-8531-da65a17409e3-ovn-node-metrics-cert\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-modprobe-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-dbus\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990855    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42c1cc67-5c1b-46b3-b120-1d89c93e832e-cni-binary-copy\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d44c172d-a3c0-4cf0-998e-4056f3fde862-iptables-alerter-script\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-device-dir\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990895    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-modprobe-d\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990946    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-run-openvswitch\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f517fb98-037e-45ae-99c0-a68b8304ef0a-sys-fs\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.990996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8bfae1f-8145-47cd-8531-da65a17409e3-host-cni-bin\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.991045    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-host-run-netns\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.991065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/785537e2-76bf-4238-9c65-cbb4ff59cf37-host\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.991266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42c1cc67-5c1b-46b3-b120-1d89c93e832e-multus-cni-dir\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:40:59.991789 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.991389    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37801522-98d0-4c13-a06a-2a700a32d141-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.993035 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.992998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/785537e2-76bf-4238-9c65-cbb4ff59cf37-etc-tuned\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:40:59.993424 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.993404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8bfae1f-8145-47cd-8531-da65a17409e3-ovn-node-metrics-cert\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.995503 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.995480    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:40:59.995727 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.995508    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:40:59.995727 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.995523    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:40:59.995727 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:40:59.995707    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:00.49569107 +0000 UTC m=+2.047930821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:40:59.997455 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.997435    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xsqj\" (UniqueName: \"kubernetes.io/projected/37801522-98d0-4c13-a06a-2a700a32d141-kube-api-access-8xsqj\") pod \"multus-additional-cni-plugins-69fgt\" (UID: \"37801522-98d0-4c13-a06a-2a700a32d141\") " pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:40:59.997540 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.997454    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hf59\" (UniqueName: \"kubernetes.io/projected/d8bfae1f-8145-47cd-8531-da65a17409e3-kube-api-access-6hf59\") pod \"ovnkube-node-89rdl\" (UID: \"d8bfae1f-8145-47cd-8531-da65a17409e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:40:59.997578 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.997564    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxglt\" (UniqueName: \"kubernetes.io/projected/445e9c17-f645-469d-b0a1-1807c3e9a08e-kube-api-access-bxglt\") pod \"node-resolver-kt2bk\" (UID: \"445e9c17-f645-469d-b0a1-1807c3e9a08e\") " pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:40:59.999804 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:40:59.999781    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bg7m\" (UniqueName: \"kubernetes.io/projected/785537e2-76bf-4238-9c65-cbb4ff59cf37-kube-api-access-7bg7m\") pod \"tuned-hg746\" (UID: \"785537e2-76bf-4238-9c65-cbb4ff59cf37\") " pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:41:00.001432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.001411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv4v9\" (UniqueName: \"kubernetes.io/projected/42c1cc67-5c1b-46b3-b120-1d89c93e832e-kube-api-access-jv4v9\") pod \"multus-4nkwr\" (UID: \"42c1cc67-5c1b-46b3-b120-1d89c93e832e\") " pod="openshift-multus/multus-4nkwr"
Jul 24 15:41:00.001862 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.001847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5q54\" (UniqueName: \"kubernetes.io/projected/f517fb98-037e-45ae-99c0-a68b8304ef0a-kube-api-access-d5q54\") pod \"aws-ebs-csi-driver-node-jnplr\" (UID: \"f517fb98-037e-45ae-99c0-a68b8304ef0a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:41:00.002043 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.002028    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr7mg\" (UniqueName: \"kubernetes.io/projected/f7afd27a-11a7-4701-9b08-408dcc75cc08-kube-api-access-hr7mg\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:00.002529 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.002515    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2nf4\" (UniqueName: \"kubernetes.io/projected/d44c172d-a3c0-4cf0-998e-4056f3fde862-kube-api-access-l2nf4\") pod \"iptables-alerter-mzlh8\" (UID: \"d44c172d-a3c0-4cf0-998e-4056f3fde862\") " pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:41:00.002637 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.002530    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b8wt\" (UniqueName: \"kubernetes.io/projected/40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5-kube-api-access-8b8wt\") pod \"node-ca-wkglg\" (UID: \"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5\") " pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:41:00.079057 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.079025    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:41:00.112280 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.112248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:41:00.123339 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.123311    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a3261e_72d6_4de2_a75a_0256b5ee1513.slice/crio-34ed44f0a37859286af059de101fa52efb1a1d7be6b658106137266cfc88d939 WatchSource:0}: Error finding container 34ed44f0a37859286af059de101fa52efb1a1d7be6b658106137266cfc88d939: Status 404 returned error can't find the container with id 34ed44f0a37859286af059de101fa52efb1a1d7be6b658106137266cfc88d939
Jul 24 15:41:00.135331 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.135301    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hg746"
Jul 24 15:41:00.140656 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.140637    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kt2bk"
Jul 24 15:41:00.144633 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.144602    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785537e2_76bf_4238_9c65_cbb4ff59cf37.slice/crio-f3180a8cbb1b52a5b68504f93b63ec2b2536673553b4e9c36505adffce16c9ae WatchSource:0}: Error finding container f3180a8cbb1b52a5b68504f93b63ec2b2536673553b4e9c36505adffce16c9ae: Status 404 returned error can't find the container with id f3180a8cbb1b52a5b68504f93b63ec2b2536673553b4e9c36505adffce16c9ae
Jul 24 15:41:00.150394 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.150371    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445e9c17_f645_469d_b0a1_1807c3e9a08e.slice/crio-381f4c212e1f4acd1db839aab096ce7d8adae1c480e6c0a1fdcc1ff82a8956b8 WatchSource:0}: Error finding container 381f4c212e1f4acd1db839aab096ce7d8adae1c480e6c0a1fdcc1ff82a8956b8: Status 404 returned error can't find the container with id 381f4c212e1f4acd1db839aab096ce7d8adae1c480e6c0a1fdcc1ff82a8956b8
Jul 24 15:41:00.151803 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.151720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkglg"
Jul 24 15:41:00.161985 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.161958    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a3161d_caec_4cb8_bdb9_cbaa63c8dcd5.slice/crio-d5c9714747b5560aeb85cf06340a3d64bccc9fa76e5fc8e995281d419257a3a8 WatchSource:0}: Error finding container d5c9714747b5560aeb85cf06340a3d64bccc9fa76e5fc8e995281d419257a3a8: Status 404 returned error can't find the container with id d5c9714747b5560aeb85cf06340a3d64bccc9fa76e5fc8e995281d419257a3a8
Jul 24 15:41:00.172834 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.172814    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nkwr"
Jul 24 15:41:00.181714 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.181684    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c1cc67_5c1b_46b3_b120_1d89c93e832e.slice/crio-22fc710ee6667a73c2adb677faf57bf042f8d099495e2cc08b472bf1a2705062 WatchSource:0}: Error finding container 22fc710ee6667a73c2adb677faf57bf042f8d099495e2cc08b472bf1a2705062: Status 404 returned error can't find the container with id 22fc710ee6667a73c2adb677faf57bf042f8d099495e2cc08b472bf1a2705062
Jul 24 15:41:00.190095 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.190070    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mzlh8"
Jul 24 15:41:00.199120 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.199097    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44c172d_a3c0_4cf0_998e_4056f3fde862.slice/crio-fc2702a8f40c23e293d80133c45487820b74a66692cf17f59482e1fb1f4217a3 WatchSource:0}: Error finding container fc2702a8f40c23e293d80133c45487820b74a66692cf17f59482e1fb1f4217a3: Status 404 returned error can't find the container with id fc2702a8f40c23e293d80133c45487820b74a66692cf17f59482e1fb1f4217a3
Jul 24 15:41:00.215570 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.215543    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr"
Jul 24 15:41:00.226254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.226228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-69fgt"
Jul 24 15:41:00.227012 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.226990    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf517fb98_037e_45ae_99c0_a68b8304ef0a.slice/crio-2ecbbe5bc0355099fde59a5ed51333444ce6f0b61d4f7b1a70754d2aa65293eb WatchSource:0}: Error finding container 2ecbbe5bc0355099fde59a5ed51333444ce6f0b61d4f7b1a70754d2aa65293eb: Status 404 returned error can't find the container with id 2ecbbe5bc0355099fde59a5ed51333444ce6f0b61d4f7b1a70754d2aa65293eb
Jul 24 15:41:00.230271 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.230218    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:00.236738 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.236713    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37801522_98d0_4c13_a06a_2a700a32d141.slice/crio-a66952c21d94eae626c348a98c7b536299b5a08c3ebc742205598c781a0edb1f WatchSource:0}: Error finding container a66952c21d94eae626c348a98c7b536299b5a08c3ebc742205598c781a0edb1f: Status 404 returned error can't find the container with id a66952c21d94eae626c348a98c7b536299b5a08c3ebc742205598c781a0edb1f
Jul 24 15:41:00.239995 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:00.239946    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8bfae1f_8145_47cd_8531_da65a17409e3.slice/crio-e14fe0f24013772e8205e92310dbd3538b7f54a0c1594dc24a51aaf51ad5c0ea WatchSource:0}: Error finding container e14fe0f24013772e8205e92310dbd3538b7f54a0c1594dc24a51aaf51ad5c0ea: Status 404 returned error can't find the container with id e14fe0f24013772e8205e92310dbd3538b7f54a0c1594dc24a51aaf51ad5c0ea
Jul 24 15:41:00.495641 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.495597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:00.495855 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.495685    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:00.495855 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.495788    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:00.495855 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.495790    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:00.496028 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.495879    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:01.495854137 +0000 UTC m=+3.048093894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:00.496028 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.495932    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:01.495919513 +0000 UTC m=+3.048159281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:00.597040 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.596994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:00.597246 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.597226    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:00.597337 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.597271    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:00.597337 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.597287    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:00.597441 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.597357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:01.597334714 +0000 UTC m=+3.149574486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:00.841518 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.841391    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:35:59 +0000 UTC" deadline="2028-05-01 19:02:45.516687962 +0000 UTC"
Jul 24 15:41:00.841518 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.841452    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15531h21m44.675244335s"
Jul 24 15:41:00.944649 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.944612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:00.944845 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:00.944762    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:00.992052 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:00.992008    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerStarted","Data":"a66952c21d94eae626c348a98c7b536299b5a08c3ebc742205598c781a0edb1f"}
Jul 24 15:41:01.007881 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.007845    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:41:01.027535 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.027479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr" event={"ID":"f517fb98-037e-45ae-99c0-a68b8304ef0a","Type":"ContainerStarted","Data":"2ecbbe5bc0355099fde59a5ed51333444ce6f0b61d4f7b1a70754d2aa65293eb"}
Jul 24 15:41:01.042717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.042671    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzlh8" event={"ID":"d44c172d-a3c0-4cf0-998e-4056f3fde862","Type":"ContainerStarted","Data":"fc2702a8f40c23e293d80133c45487820b74a66692cf17f59482e1fb1f4217a3"}
Jul 24 15:41:01.057556 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.057332    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nkwr" event={"ID":"42c1cc67-5c1b-46b3-b120-1d89c93e832e","Type":"ContainerStarted","Data":"22fc710ee6667a73c2adb677faf57bf042f8d099495e2cc08b472bf1a2705062"}
Jul 24 15:41:01.080142 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.080079    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kt2bk" event={"ID":"445e9c17-f645-469d-b0a1-1807c3e9a08e","Type":"ContainerStarted","Data":"381f4c212e1f4acd1db839aab096ce7d8adae1c480e6c0a1fdcc1ff82a8956b8"}
Jul 24 15:41:01.092337 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.092238    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hg746" event={"ID":"785537e2-76bf-4238-9c65-cbb4ff59cf37","Type":"ContainerStarted","Data":"f3180a8cbb1b52a5b68504f93b63ec2b2536673553b4e9c36505adffce16c9ae"}
Jul 24 15:41:01.095050 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.095015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"e14fe0f24013772e8205e92310dbd3538b7f54a0c1594dc24a51aaf51ad5c0ea"}
Jul 24 15:41:01.112717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.112680    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkglg" event={"ID":"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5","Type":"ContainerStarted","Data":"d5c9714747b5560aeb85cf06340a3d64bccc9fa76e5fc8e995281d419257a3a8"}
Jul 24 15:41:01.142409 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.142365    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wxwm" event={"ID":"53a3261e-72d6-4de2-a75a-0256b5ee1513","Type":"ContainerStarted","Data":"34ed44f0a37859286af059de101fa52efb1a1d7be6b658106137266cfc88d939"}
Jul 24 15:41:01.505869 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.505806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:01.506076 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.505877    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:01.506076 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.506023    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:01.506226 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.506085    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:03.506067671 +0000 UTC m=+5.058307437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:01.506226 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.506223    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:01.506356 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.506276    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:03.506257842 +0000 UTC m=+5.058497598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:01.607896 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.607306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:01.607896 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.607474    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:01.607896 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.607494    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:01.607896 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.607508    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:01.607896 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.607565    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:03.607545196 +0000 UTC m=+5.159784969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:01.723370 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.723336    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:41:01.842267 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.842164    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:35:59 +0000 UTC" deadline="2028-03-30 11:46:31.290454452 +0000 UTC"
Jul 24 15:41:01.842267 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.842212    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14756h5m29.448245207s"
Jul 24 15:41:01.944277 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.943578    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:01.944277 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.943707    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:01.944277 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:01.944114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:01.944277 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:01.944232    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:02.945609 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:02.945576    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:02.946111 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:02.945716    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:03.526199 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:03.526161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:03.526381 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:03.526213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:03.526381 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.526378    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:03.526499 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.526439    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:07.526420137 +0000 UTC m=+9.078659893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:03.526922 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.526811    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:03.526922 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.526855    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:07.526843438 +0000 UTC m=+9.079083188 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:03.627535 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:03.627490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:03.627730 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.627715    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:03.627793 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.627740    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:03.627793 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.627754    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:03.627895 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.627815    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:07.627795462 +0000 UTC m=+9.180035229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:03.943928 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:03.943189    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:03.943928 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.943336    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:03.943928 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:03.943728    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:03.943928 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:03.943827    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:04.944377 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:04.944342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:04.944819 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:04.944494    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:05.943521 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:05.943483    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:05.943717 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:05.943638    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:05.944042 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:05.944023    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:05.944163 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:05.944147    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:06.945014 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:06.944982    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:06.945409 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:06.945143    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:07.560905 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:07.560859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:07.561089 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:07.560915    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:07.561194 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.561088    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:07.561194 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.561171    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:15.561151517 +0000 UTC m=+17.113391285 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:07.561580 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.561557    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:07.561710 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.561609    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:15.561594733 +0000 UTC m=+17.113834488 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:07.661524 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:07.661479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:07.661722 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.661694    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:07.661722 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.661711    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:07.661722 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.661723    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:07.661873 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.661784    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:15.661764705 +0000 UTC m=+17.214004460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:07.944251 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:07.943771    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:07.944251 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.943890    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:07.944251 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:07.943943    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:07.944251 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:07.944039    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:08.944187 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:08.944146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:08.944598 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:08.944281    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:09.943072 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:09.943038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:09.943254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:09.943039    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:09.943254 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:09.943200    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:09.943380 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:09.943244    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:10.943354 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:10.943317    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:10.943780 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:10.943456    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:11.943309 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:11.943273    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:11.943481 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:11.943273    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:11.943481 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:11.943416    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:11.943481 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:11.943454    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:12.943702 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:12.943668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:12.944165 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:12.943802    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:13.942876 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:13.942841    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:13.942876 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:13.942877    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:13.943161 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:13.942984    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:13.943230 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:13.943159    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:14.943195 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:14.943140    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:14.943577 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:14.943277    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:15.624202 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:15.624150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:15.624202 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:15.624207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:15.624469 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.624324    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:15.624469 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.624327    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:15.624469 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.624392    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:31.624371677 +0000 UTC m=+33.176611434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:15.624469 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.624410    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:41:31.624400441 +0000 UTC m=+33.176640209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:15.725378 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:15.725339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:15.725565 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.725547    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:15.725615 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.725578    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:15.725615 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.725591    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:15.725698 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.725657    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:41:31.725638471 +0000 UTC m=+33.277878224 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:15.943212 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:15.943112    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:15.943654 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:15.943118    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:15.943654 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.943265    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:15.943654 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:15.943339    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:16.944007 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:16.943813    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:16.944517 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:16.944114    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:17.943837 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:17.943793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:17.943837 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:17.943827    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:17.944086 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:17.943920    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:17.944086 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:17.944054    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:18.944363 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:18.944313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:18.944842 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:18.944433    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:19.943528 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:19.943493    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:19.943528 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:19.943539    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:19.943757 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:19.943640    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:19.943808 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:19.943755    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:20.943121 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:20.942902    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:20.943671 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:20.943270    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:21.193492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.193452    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkglg" event={"ID":"40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5","Type":"ContainerStarted","Data":"3f44183565e5c01611e7ed8b62d1d8766b719305eec2aa4013532195d3ee7660"}
Jul 24 15:41:21.198253 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.197693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7wxwm" event={"ID":"53a3261e-72d6-4de2-a75a-0256b5ee1513","Type":"ContainerStarted","Data":"5e8c10ab500fa15eb537662e09e685f07ec57276ec559398a85d4b5cff45f019"}
Jul 24 15:41:21.199829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.199758    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal" event={"ID":"4fd32fde91ac02b5f7cfbf50c6d4bcc8","Type":"ContainerStarted","Data":"c5318230795840b611f0755230b8d68f3a5e169ad2b9e1431d3c8d87314ef225"}
Jul 24 15:41:21.201725 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.201697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal" event={"ID":"2d0a1fc27b2b71d2f6d0b5866eb66fcf","Type":"ContainerStarted","Data":"97597a5634e6b72ebf18524c12fb9272f4d4d9fde858198cb354bf78979ab543"}
Jul 24 15:41:21.203907 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.203883    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerStarted","Data":"7589fc02bb8ff6207ef6285363b8506ca8cabed4f48612f8468d3d38acf43166"}
Jul 24 15:41:21.206280 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.206255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr" event={"ID":"f517fb98-037e-45ae-99c0-a68b8304ef0a","Type":"ContainerStarted","Data":"c054066f0cdcd954811bbaffc68c155402fe54ff4d8c0a15099c9ca9ddb94483"}
Jul 24 15:41:21.207943 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.207917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nkwr" event={"ID":"42c1cc67-5c1b-46b3-b120-1d89c93e832e","Type":"ContainerStarted","Data":"91353771ffd5e130037e7a4b8c9d9cb08312290f2ee4dd0a639d5b631b86c36a"}
Jul 24 15:41:21.209587 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.209564    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kt2bk" event={"ID":"445e9c17-f645-469d-b0a1-1807c3e9a08e","Type":"ContainerStarted","Data":"bbf4e472265593dff2d9e4b79c685c3c7bafbb7d10dcff25ab065b4d551538c9"}
Jul 24 15:41:21.212402 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.212345    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hg746" event={"ID":"785537e2-76bf-4238-9c65-cbb4ff59cf37","Type":"ContainerStarted","Data":"32d47db4b398cd8a69819e66e50affbe816b05d25531c5b261f5bee064e3f89e"}
Jul 24 15:41:21.215617 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.215590    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"f9861acba1205608d18069a931dddb3cc17ec449eb9b05a36727964e518511ab"}
Jul 24 15:41:21.215725 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.215676    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"2068ba1137c58f1428157150d8d605367039b08bc73db155796aeb7f1d00a12f"}
Jul 24 15:41:21.215900 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.215730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"6364443c1256133a3f716be072cb4fa75459d7086ab78e47922f3c5c78c5f22d"}
Jul 24 15:41:21.215900 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.215788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"5234f3bf53f84df9ce3dbb9677e63734cecabcbbf2f57666b7671c6f13eba66f"}
Jul 24 15:41:21.215900 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.215838    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"6016aa98b1ab352efe2261a0b2808af280949bdd816e6c2fb089e963f69ad632"}
Jul 24 15:41:21.226076 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.226034    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-134.ec2.internal" podStartSLOduration=22.226019533 podStartE2EDuration="22.226019533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:21.225814651 +0000 UTC m=+22.778054424" watchObservedRunningTime="2026-07-24 15:41:21.226019533 +0000 UTC m=+22.778259306"
Jul 24 15:41:21.226221 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.226177    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7wxwm" podStartSLOduration=1.922551528 podStartE2EDuration="22.226168295s" totalImagesPullingTime="20.303616767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:21.21234945 +0000 UTC m=+22.764589223" watchObservedRunningTime="2026-07-24 15:41:21.226168295 +0000 UTC m=+22.778408071"
Jul 24 15:41:21.308319 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.308265    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kt2bk" podStartSLOduration=2.031570594 podStartE2EDuration="22.308247576s" totalImagesPullingTime="20.276676982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:21.29175306 +0000 UTC m=+22.843992845" watchObservedRunningTime="2026-07-24 15:41:21.308247576 +0000 UTC m=+22.860487349"
Jul 24 15:41:21.308476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.308363    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4nkwr" podStartSLOduration=1.990944089 podStartE2EDuration="22.308359088s" totalImagesPullingTime="20.317414999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:21.307812513 +0000 UTC m=+22.860052285" watchObservedRunningTime="2026-07-24 15:41:21.308359088 +0000 UTC m=+22.860598885"
Jul 24 15:41:21.327051 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.327006    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hg746" podStartSLOduration=2.0092127 podStartE2EDuration="22.32699s" totalImagesPullingTime="20.3177773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:21.326715038 +0000 UTC m=+22.878954811" watchObservedRunningTime="2026-07-24 15:41:21.32699 +0000 UTC m=+22.879229773"
Jul 24 15:41:21.451544 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.451505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:21.524345 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.524307    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:21.836071 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.836040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:21.891526 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.891494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:21.943517 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.943322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:21.944146 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:21.943342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:21.944146 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:21.943629    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:21.944146 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:21.943770    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:22.220157 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.220011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"4a6590ad89f12354db0803c346ff6b44232ec6f54aa355c2c120bdc45da6fb9f"}
Jul 24 15:41:22.221398 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.221372    2567 generic.go:356] "Generic (PLEG): container finished" podID="2d0a1fc27b2b71d2f6d0b5866eb66fcf" containerID="97597a5634e6b72ebf18524c12fb9272f4d4d9fde858198cb354bf78979ab543" exitCode=0
Jul 24 15:41:22.221518 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.221440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal" event={"ID":"2d0a1fc27b2b71d2f6d0b5866eb66fcf","Type":"ContainerDied","Data":"97597a5634e6b72ebf18524c12fb9272f4d4d9fde858198cb354bf78979ab543"}
Jul 24 15:41:22.222756 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.222733    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="7589fc02bb8ff6207ef6285363b8506ca8cabed4f48612f8468d3d38acf43166" exitCode=0
Jul 24 15:41:22.222860 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.222815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"7589fc02bb8ff6207ef6285363b8506ca8cabed4f48612f8468d3d38acf43166"}
Jul 24 15:41:22.224394 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.224368    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mzlh8" event={"ID":"d44c172d-a3c0-4cf0-998e-4056f3fde862","Type":"ContainerStarted","Data":"0aaa944ac9ec457a3d8e5e97ec3a255027ae34de751b2c1e6fef34832e37ce20"}
Jul 24 15:41:22.249264 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.249213    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkglg" podStartSLOduration=2.952343268 podStartE2EDuration="23.249192504s" totalImagesPullingTime="20.296849236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:22.248627162 +0000 UTC m=+23.800866934" watchObservedRunningTime="2026-07-24 15:41:22.249192504 +0000 UTC m=+23.801432278"
Jul 24 15:41:22.281889 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.281839    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mzlh8" podStartSLOduration=3.022275259 podStartE2EDuration="23.28182197s" totalImagesPullingTime="20.259546711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:22.281599668 +0000 UTC m=+23.833839441" watchObservedRunningTime="2026-07-24 15:41:22.28182197 +0000 UTC m=+23.834061742"
Jul 24 15:41:22.400615 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.400581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:22.455178 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.455149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:22.580502 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.580459    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 15:41:22.731893 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.731802    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:41:22.732556 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.732526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:41:22.842977 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.842827    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:41:22.580488951Z","UUID":"6e042111-c800-418b-9bc9-aaa2d5adf0b9","Handler":null,"Name":"","Endpoint":""}
Jul 24 15:41:22.844717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.844687    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 15:41:22.844873 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.844736    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 15:41:22.943443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:22.943408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:22.943673 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:22.943557    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:23.230049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.229988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr" event={"ID":"f517fb98-037e-45ae-99c0-a68b8304ef0a","Type":"ContainerStarted","Data":"a18ce63ce5c327705580460f90ac1d774487531c35895a6ad7b803ae03bfb838"}
Jul 24 15:41:23.233418 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.233375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal" event={"ID":"2d0a1fc27b2b71d2f6d0b5866eb66fcf","Type":"ContainerStarted","Data":"1101b4208240649c5c728e4e8b37fa0d65843715506d5f50b4517b426b5a564e"}
Jul 24 15:41:23.237380 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.234234    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:41:23.237380 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.234327    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7wxwm"
Jul 24 15:41:23.263954 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.263843    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-134.ec2.internal" podStartSLOduration=24.263819868 podStartE2EDuration="24.263819868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:23.248690937 +0000 UTC m=+24.800930713" watchObservedRunningTime="2026-07-24 15:41:23.263819868 +0000 UTC m=+24.816059642"
Jul 24 15:41:23.308056 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.308018    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:23.943070 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.943034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:23.943267 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:23.943206    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:23.943267 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:23.943238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:23.943406 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:23.943371    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:24.237626 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:24.237592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr" event={"ID":"f517fb98-037e-45ae-99c0-a68b8304ef0a","Type":"ContainerStarted","Data":"988bbdf4dda80a8cad94d0f6e5791957cc6e3093020a12cfdf4048a0c73062da"}
Jul 24 15:41:24.241347 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:24.241305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"93e4a0cb9878df44fee76c51716d160ce50eb6a9a052d03d161a241f72b5bc66"}
Jul 24 15:41:24.254773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:24.254677    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jnplr" podStartSLOduration=2.581703009 podStartE2EDuration="25.254655145s" totalImagesPullingTime="22.672952136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:24.254454308 +0000 UTC m=+25.806694059" watchObservedRunningTime="2026-07-24 15:41:24.254655145 +0000 UTC m=+25.806894914"
Jul 24 15:41:24.943687 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:24.943652    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:24.944009 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:24.943944    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:25.943584 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:25.943501    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:25.943584 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:25.943544    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:25.944071 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:25.943638    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:25.944071 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:25.943771    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:26.944154 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:26.943978    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:26.944560 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:26.944251    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:27.251342 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.251302    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" event={"ID":"d8bfae1f-8145-47cd-8531-da65a17409e3","Type":"ContainerStarted","Data":"4998031ccb63051eaa3d0db7fda667c88b88c4268c9d5fd13cf37800329cb437"}
Jul 24 15:41:27.251575 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.251556    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:27.251697 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.251633    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:27.254311 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.254270    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="bdbc373d686c1f3cb131ae11c7370abea541b01d0a8e9fc250e184a29d651213" exitCode=0
Jul 24 15:41:27.254446 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.254309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"bdbc373d686c1f3cb131ae11c7370abea541b01d0a8e9fc250e184a29d651213"}
Jul 24 15:41:27.269775 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.269749    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:27.277394 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.277339    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl" podStartSLOduration=7.86868938 podStartE2EDuration="28.277323599s" totalImagesPullingTime="20.408634219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:27.276384191 +0000 UTC m=+28.828623965" watchObservedRunningTime="2026-07-24 15:41:27.277323599 +0000 UTC m=+28.829563372"
Jul 24 15:41:27.943537 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.943501    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:27.943720 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:27.943637    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:27.944312 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:27.944288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:27.944748 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:27.944408    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:28.256903 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.256868    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:28.273010 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.272977    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:41:28.727659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.727461    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-59zdt"]
Jul 24 15:41:28.727838 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.727807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:28.727927 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:28.727902    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:28.728228 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.728192    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jbfbj"]
Jul 24 15:41:28.728366 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.728352    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:28.728453 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:28.728437    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:28.730369 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.730342    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-44n2k"]
Jul 24 15:41:28.730468 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:28.730455    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:28.730580 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:28.730552    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:29.260344 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.260306    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="01eff876b0c1ebe0f30de5f5e2da8de9dbfe941e7efba8a2ad30493dc4776c2b" exitCode=0
Jul 24 15:41:29.260825 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.260390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"01eff876b0c1ebe0f30de5f5e2da8de9dbfe941e7efba8a2ad30493dc4776c2b"}
Jul 24 15:41:29.550938 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.550859    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:29.626474 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.626439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:29.723655 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.723624    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:29.815472 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.815378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:29.943190 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.943155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:29.943354 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:29.943325    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:29.987056 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:29.987015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:30.042829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:30.042797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:30.943255 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:30.943220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:30.943614 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:30.943220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:30.943614 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:30.943339    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:30.943614 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:30.943399    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:31.224731 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.224635    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:31.267994 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.267957    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="b7169c436e42b7ab39572ad168871867a1764c39300040e6b2c6ecbe513f889c" exitCode=0
Jul 24 15:41:31.267994 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.268005    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"b7169c436e42b7ab39572ad168871867a1764c39300040e6b2c6ecbe513f889c"}
Jul 24 15:41:31.323117 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.323078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:31.424427 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.424387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:31.524953 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.524917    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:31.640797 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.640751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:31.642674 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.642654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:31.642770 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.642686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:31.642812 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.642799    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:31.642867 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.642858    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret podName:44fcb14e-69e3-4b41-8baa-ccb7b8f4793d nodeName:}" failed. No retries permitted until 2026-07-24 15:42:03.642845095 +0000 UTC m=+65.195084846 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret") pod "global-pull-secret-syncer-59zdt" (UID: "44fcb14e-69e3-4b41-8baa-ccb7b8f4793d") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:41:31.642907 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.642799    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:31.642936 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.642919    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs podName:f7afd27a-11a7-4701-9b08-408dcc75cc08 nodeName:}" failed. No retries permitted until 2026-07-24 15:42:03.642906995 +0000 UTC m=+65.195146746 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs") pod "network-metrics-daemon-jbfbj" (UID: "f7afd27a-11a7-4701-9b08-408dcc75cc08") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:41:31.743886 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.743851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:31.744027 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.743968    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:41:31.744027 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.743981    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:41:31.744027 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.743990    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zh4p9 for pod openshift-network-diagnostics/network-check-target-44n2k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:31.744111 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.744037    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9 podName:3eb29b7c-fcff-4c45-9561-1bf24918177b nodeName:}" failed. No retries permitted until 2026-07-24 15:42:03.744023146 +0000 UTC m=+65.296262896 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zh4p9" (UniqueName: "kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9") pod "network-check-target-44n2k" (UID: "3eb29b7c-fcff-4c45-9561-1bf24918177b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:41:31.943781 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:31.943686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:31.944259 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:31.943838    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:32.275892 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.275857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:32.327074 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.327031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:32.674423 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.674346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:32.728121 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.728083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:32.943702 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.943607    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:32.943923 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:32.943613    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:32.943923 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:32.943753    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-44n2k" podUID="3eb29b7c-fcff-4c45-9561-1bf24918177b"
Jul 24 15:41:32.943923 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:32.943835    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jbfbj" podUID="f7afd27a-11a7-4701-9b08-408dcc75cc08"
Jul 24 15:41:33.943396 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:33.943363    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:33.943678 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:33.943500    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-59zdt" podUID="44fcb14e-69e3-4b41-8baa-ccb7b8f4793d"
Jul 24 15:41:34.232030 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.231939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:34.283780 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.283752    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-134.ec2.internal" event="NodeReady"
Jul 24 15:41:34.283963 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.283906    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 15:41:34.333147 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.333076    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:41:34.337349 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.337315    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-w87bk"]
Jul 24 15:41:34.337514 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.337447    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.340368 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.340343    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n2cf7"]
Jul 24 15:41:34.340526 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.340481    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.341602 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.341578    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 15:41:34.341839 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.341824    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 15:41:34.342419 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.342402    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 15:41:34.342617 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.342603    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wlmxz\""
Jul 24 15:41:34.343343 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.343321    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.343549 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.343533    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 15:41:34.344496 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.344103    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m8m7j\""
Jul 24 15:41:34.344496 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.344224    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 15:41:34.344496 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.344234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 15:41:34.346840 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.346819    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 15:41:34.347070 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.347052    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dz9d4\""
Jul 24 15:41:34.347354 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.347335    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 15:41:34.347496 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.347478    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 15:41:34.348201 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.348180    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2cf7"]
Jul 24 15:41:34.353946 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.352693    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w87bk"]
Jul 24 15:41:34.353946 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.352756    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:41:34.455303 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.455274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:34.462200 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462200 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-config-volume\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.462432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462337    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-cert\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.462432 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-tmp-dir\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8jnf\" (UniqueName: \"kubernetes.io/projected/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-kube-api-access-x8jnf\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hpcl\" (UniqueName: \"kubernetes.io/projected/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-kube-api-access-7hpcl\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.462659 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.462652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5m6\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.511333 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.511241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:34.563043 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8jnf\" (UniqueName: \"kubernetes.io/projected/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-kube-api-access-x8jnf\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.563043 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hpcl\" (UniqueName: \"kubernetes.io/projected/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-kube-api-access-7hpcl\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.563306 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563306 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5m6\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563403 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563403 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563335    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563403 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563403 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-config-volume\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.563532 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563532 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563499    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-cert\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.563532 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563521    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563619 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.563619 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563602    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-tmp-dir\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.563687 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.563622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.563809 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:34.563788    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:41:34.563883 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:34.563811    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78df859bd7-tc8gh: secret "image-registry-tls" not found
Jul 24 15:41:34.563883 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:34.563869    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls podName:cae858f5-b847-4d94-9e20-224f444bc494 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:35.063848757 +0000 UTC m=+36.616088509 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls") pod "image-registry-78df859bd7-tc8gh" (UID: "cae858f5-b847-4d94-9e20-224f444bc494") : secret "image-registry-tls" not found
Jul 24 15:41:34.564037 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.564016    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.564162 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:34.564115    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:41:34.564287 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:34.564196    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls podName:9a52f9e9-355e-4e1e-93ab-757fbaee85d6 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:35.064175468 +0000 UTC m=+36.616415221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls") pod "dns-default-n2cf7" (UID: "9a52f9e9-355e-4e1e-93ab-757fbaee85d6") : secret "dns-default-metrics-tls" not found
Jul 24 15:41:34.564375 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.564356    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-tmp-dir\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.564499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.564475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-config-volume\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.564996 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.564971    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.565404 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.565381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.569302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.569278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.569421 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.569285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-cert\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.569645 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.569614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.575722 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.575271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8jnf\" (UniqueName: \"kubernetes.io/projected/54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab-kube-api-access-x8jnf\") pod \"ingress-canary-w87bk\" (UID: \"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab\") " pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.575722 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.575321    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5m6\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.575722 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.575464    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hpcl\" (UniqueName: \"kubernetes.io/projected/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-kube-api-access-7hpcl\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:34.575722 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.575679    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:34.664743 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.664705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-w87bk"
Jul 24 15:41:34.943994 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.943901    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:41:34.944186 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.943901    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:41:34.946767 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.946736    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:41:34.946941 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.946877    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nh5k9\""
Jul 24 15:41:34.946941 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.946891    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:41:34.947043 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.947005    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvmsc\""
Jul 24 15:41:34.947593 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:34.947572    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:41:35.067965 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:35.067913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:35.068198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:35.068002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:35.068198 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:35.068147    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:41:35.068198 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:35.068172    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78df859bd7-tc8gh: secret "image-registry-tls" not found
Jul 24 15:41:35.068366 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:35.068259    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls podName:cae858f5-b847-4d94-9e20-224f444bc494 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:36.068236178 +0000 UTC m=+37.620475936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls") pod "image-registry-78df859bd7-tc8gh" (UID: "cae858f5-b847-4d94-9e20-224f444bc494") : secret "image-registry-tls" not found
Jul 24 15:41:35.068492 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:35.068465    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:41:35.068629 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:35.068533    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls podName:9a52f9e9-355e-4e1e-93ab-757fbaee85d6 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:36.068516955 +0000 UTC m=+37.620756706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls") pod "dns-default-n2cf7" (UID: "9a52f9e9-355e-4e1e-93ab-757fbaee85d6") : secret "dns-default-metrics-tls" not found
Jul 24 15:41:35.747208 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:35.747172    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-w87bk"]
Jul 24 15:41:35.754247 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:35.754213    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d7cb73_8e96_4b36_ac2e_9e0d8bc48eab.slice/crio-bb8e697ee25d72fbc817f9a6146f45051567278525c756878e2251f983a71d7b WatchSource:0}: Error finding container bb8e697ee25d72fbc817f9a6146f45051567278525c756878e2251f983a71d7b: Status 404 returned error can't find the container with id bb8e697ee25d72fbc817f9a6146f45051567278525c756878e2251f983a71d7b
Jul 24 15:41:35.943102 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:35.943061    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:41:35.945696 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:35.945504    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:41:36.076182 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:36.076113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:36.076378 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:36.076228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:36.076378 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:36.076269    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:41:36.076378 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:36.076289    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78df859bd7-tc8gh: secret "image-registry-tls" not found
Jul 24 15:41:36.076378 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:36.076356    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls podName:cae858f5-b847-4d94-9e20-224f444bc494 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:38.076338583 +0000 UTC m=+39.628578337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls") pod "image-registry-78df859bd7-tc8gh" (UID: "cae858f5-b847-4d94-9e20-224f444bc494") : secret "image-registry-tls" not found
Jul 24 15:41:36.076378 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:36.076376    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:41:36.076615 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:36.076424    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls podName:9a52f9e9-355e-4e1e-93ab-757fbaee85d6 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:38.076409437 +0000 UTC m=+39.628649188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls") pod "dns-default-n2cf7" (UID: "9a52f9e9-355e-4e1e-93ab-757fbaee85d6") : secret "dns-default-metrics-tls" not found
Jul 24 15:41:36.280877 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:36.280841    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w87bk" event={"ID":"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab","Type":"ContainerStarted","Data":"bb8e697ee25d72fbc817f9a6146f45051567278525c756878e2251f983a71d7b"}
Jul 24 15:41:36.284285 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:36.284245    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="cc9ae9c42542fb3d0156aaa61b24a32b16896776e49be4cb6eb1d7a2a0667760" exitCode=0
Jul 24 15:41:36.284447 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:36.284310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"cc9ae9c42542fb3d0156aaa61b24a32b16896776e49be4cb6eb1d7a2a0667760"}
Jul 24 15:41:37.289882 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:37.289845    2567 generic.go:356] "Generic (PLEG): container finished" podID="37801522-98d0-4c13-a06a-2a700a32d141" containerID="c5a06e632c64f8e34c201244a2dd4e3e3a786e6622b32c91031dc22bcc7af1c1" exitCode=0
Jul 24 15:41:37.290290 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:37.289927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerDied","Data":"c5a06e632c64f8e34c201244a2dd4e3e3a786e6622b32c91031dc22bcc7af1c1"}
Jul 24 15:41:37.425984 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:37.425955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:37.525412 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:37.525372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:37.625407 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:37.625357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.092312 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.092270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:38.092503 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.092377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:38.092503 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.092426    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:41:38.092503 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.092496    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls podName:9a52f9e9-355e-4e1e-93ab-757fbaee85d6 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.092481442 +0000 UTC m=+43.644721194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls") pod "dns-default-n2cf7" (UID: "9a52f9e9-355e-4e1e-93ab-757fbaee85d6") : secret "dns-default-metrics-tls" not found
Jul 24 15:41:38.092662 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.092536    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:41:38.092662 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.092551    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78df859bd7-tc8gh: secret "image-registry-tls" not found
Jul 24 15:41:38.092662 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.092601    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls podName:cae858f5-b847-4d94-9e20-224f444bc494 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.092585493 +0000 UTC m=+43.644825247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls") pod "image-registry-78df859bd7-tc8gh" (UID: "cae858f5-b847-4d94-9e20-224f444bc494") : secret "image-registry-tls" not found
Jul 24 15:41:38.136775 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.136737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.203027 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.202994    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.295910 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.295869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-69fgt" event={"ID":"37801522-98d0-4c13-a06a-2a700a32d141","Type":"ContainerStarted","Data":"2adb51649889431ae9f2e98a7185fc016783ae92e7ee5438708fdf71272d5574"}
Jul 24 15:41:38.324731 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.324678    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-69fgt" podStartSLOduration=8.063482636 podStartE2EDuration="39.324656897s" totalImagesPullingTime="31.261174261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:38.322015775 +0000 UTC m=+39.874255595" watchObservedRunningTime="2026-07-24 15:41:38.324656897 +0000 UTC m=+39.876896673"
Jul 24 15:41:38.428574 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.428476    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.434407 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.434376    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fkfp2"]
Jul 24 15:41:38.457472 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.457443    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkfp2"]
Jul 24 15:41:38.457638 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.457559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.460161 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.460116    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 15:41:38.460524 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.460508    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 15:41:38.460612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.460533    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 15:41:38.460612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.460546    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 15:41:38.460612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.460557    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qtzxv\""
Jul 24 15:41:38.556982 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.556948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.596064 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.595989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.596064 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.596025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62ba8453-57e4-4f6f-aee0-73dd75259434-data-volume\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.596064 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.596044    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.596064 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.596064    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-598pw\" (UniqueName: \"kubernetes.io/projected/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-api-access-598pw\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.596384 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.596250    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62ba8453-57e4-4f6f-aee0-73dd75259434-crio-socket\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.657853 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.657802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.696726 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62ba8453-57e4-4f6f-aee0-73dd75259434-crio-socket\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.696726 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696672    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.696726 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62ba8453-57e4-4f6f-aee0-73dd75259434-data-volume\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.696996 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.696787    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:41:38.696996 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.696872    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:39.196849974 +0000 UTC m=+40.749089728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : configmap "kube-rbac-proxy" not found
Jul 24 15:41:38.696996 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/62ba8453-57e4-4f6f-aee0-73dd75259434-crio-socket\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.696996 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696956    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.697241 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.696998    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-598pw\" (UniqueName: \"kubernetes.io/projected/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-api-access-598pw\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.697241 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.697037    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/62ba8453-57e4-4f6f-aee0-73dd75259434-data-volume\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.697241 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.697049    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:38.697241 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:38.697149    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:39.197111765 +0000 UTC m=+40.749351518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:38.708254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.708114    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-598pw\" (UniqueName: \"kubernetes.io/projected/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-api-access-598pw\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:38.907279 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.907246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:38.965873 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:38.965787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:39.200981 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.200940    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:39.200981 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.200986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:39.201245 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:39.201070    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:41:39.201245 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:39.201117    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:39.201245 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:39.201166    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:40.201149476 +0000 UTC m=+41.753389227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : configmap "kube-rbac-proxy" not found
Jul 24 15:41:39.201245 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:39.201185    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:40.201173306 +0000 UTC m=+41.753413057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:39.299951 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.299914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-w87bk" event={"ID":"54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab","Type":"ContainerStarted","Data":"26dfe57da9bf9fda6c5516e4faaacf0f8b43712c5e31c017a123eb8938c977fc"}
Jul 24 15:41:39.315494 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.315442    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-w87bk" podStartSLOduration=2.5234983399999997 podStartE2EDuration="5.31542453s" totalImagesPullingTime="2.79192619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:39.314563674 +0000 UTC m=+40.866803455" watchObservedRunningTime="2026-07-24 15:41:39.31542453 +0000 UTC m=+40.867664302"
Jul 24 15:41:39.504378 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.504345    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"]
Jul 24 15:41:39.534727 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.534692    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:39.543150 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.543089    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"]
Jul 24 15:41:39.543301 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.543241    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"
Jul 24 15:41:39.546414 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.546387    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 15:41:39.546572 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.546415    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 15:41:39.546572 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.546496    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-45mtl\""
Jul 24 15:41:39.634715 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.634618    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:39.706307 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.706272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpw62\" (UniqueName: \"kubernetes.io/projected/fd79b9a9-991e-46a5-b5b0-2cc6710106f0-kube-api-access-gpw62\") pod \"migrator-6865c9967-m7lh8\" (UID: \"fd79b9a9-991e-46a5-b5b0-2cc6710106f0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"
Jul 24 15:41:39.715063 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.715021    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-t8zmz"]
Jul 24 15:41:39.734273 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.734239    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-t8zmz"]
Jul 24 15:41:39.734414 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.734398    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.737146 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.737095    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 15:41:39.737280 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.737104    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 15:41:39.737280 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.737167    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 15:41:39.737429 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.737415    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 15:41:39.737495 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.737464    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qfwks\""
Jul 24 15:41:39.749893 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.749844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:39.804580 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.804543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:39.807463 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.807444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpw62\" (UniqueName: \"kubernetes.io/projected/fd79b9a9-991e-46a5-b5b0-2cc6710106f0-kube-api-access-gpw62\") pod \"migrator-6865c9967-m7lh8\" (UID: \"fd79b9a9-991e-46a5-b5b0-2cc6710106f0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"
Jul 24 15:41:39.807581 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.807496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/824a444c-91e2-48d9-b200-dc173ca76f82-signing-key\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.807639 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.807621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/824a444c-91e2-48d9-b200-dc173ca76f82-signing-cabundle\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.807711 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.807696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77zs7\" (UniqueName: \"kubernetes.io/projected/824a444c-91e2-48d9-b200-dc173ca76f82-kube-api-access-77zs7\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.816188 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.816153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpw62\" (UniqueName: \"kubernetes.io/projected/fd79b9a9-991e-46a5-b5b0-2cc6710106f0-kube-api-access-gpw62\") pod \"migrator-6865c9967-m7lh8\" (UID: \"fd79b9a9-991e-46a5-b5b0-2cc6710106f0\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"
Jul 24 15:41:39.852244 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.852212    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"
Jul 24 15:41:39.908350 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.908260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/824a444c-91e2-48d9-b200-dc173ca76f82-signing-cabundle\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.908350 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.908321    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77zs7\" (UniqueName: \"kubernetes.io/projected/824a444c-91e2-48d9-b200-dc173ca76f82-kube-api-access-77zs7\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.908565 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.908395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/824a444c-91e2-48d9-b200-dc173ca76f82-signing-key\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.908913 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.908893    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/824a444c-91e2-48d9-b200-dc173ca76f82-signing-cabundle\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.910894 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.910875    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/824a444c-91e2-48d9-b200-dc173ca76f82-signing-key\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:39.918016 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:39.917991    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77zs7\" (UniqueName: \"kubernetes.io/projected/824a444c-91e2-48d9-b200-dc173ca76f82-kube-api-access-77zs7\") pod \"service-ca-658d7b7c78-t8zmz\" (UID: \"824a444c-91e2-48d9-b200-dc173ca76f82\") " pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:40.002683 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.002651    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8"]
Jul 24 15:41:40.005683 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:40.005656    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd79b9a9_991e_46a5_b5b0_2cc6710106f0.slice/crio-36b343e7982090efbd9f3b85102ca7a163115b120234c42a099e509443fbe893 WatchSource:0}: Error finding container 36b343e7982090efbd9f3b85102ca7a163115b120234c42a099e509443fbe893: Status 404 returned error can't find the container with id 36b343e7982090efbd9f3b85102ca7a163115b120234c42a099e509443fbe893
Jul 24 15:41:40.032040 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.032014    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kt2bk_445e9c17-f645-469d-b0a1-1807c3e9a08e/dns-node-resolver/0.log"
Jul 24 15:41:40.043355 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.043329    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz"
Jul 24 15:41:40.211716 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.211617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:40.211716 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.211668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:40.211929 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:40.211744    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:41:40.211929 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:40.211798    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:40.211929 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:40.211815    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.211797933 +0000 UTC m=+43.764037683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : configmap "kube-rbac-proxy" not found
Jul 24 15:41:40.211929 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:40.211851    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.211834375 +0000 UTC m=+43.764074131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:40.217688 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.217656    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-t8zmz"]
Jul 24 15:41:40.221373 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:40.221349    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824a444c_91e2_48d9_b200_dc173ca76f82.slice/crio-621a631415b865acf40da1d645e53fc4ca07a4f830e3f4b5388893e17587b94c WatchSource:0}: Error finding container 621a631415b865acf40da1d645e53fc4ca07a4f830e3f4b5388893e17587b94c: Status 404 returned error can't find the container with id 621a631415b865acf40da1d645e53fc4ca07a4f830e3f4b5388893e17587b94c
Jul 24 15:41:40.303287 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.303245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz" event={"ID":"824a444c-91e2-48d9-b200-dc173ca76f82","Type":"ContainerStarted","Data":"621a631415b865acf40da1d645e53fc4ca07a4f830e3f4b5388893e17587b94c"}
Jul 24 15:41:40.304411 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.304384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8" event={"ID":"fd79b9a9-991e-46a5-b5b0-2cc6710106f0","Type":"ContainerStarted","Data":"36b343e7982090efbd9f3b85102ca7a163115b120234c42a099e509443fbe893"}
Jul 24 15:41:40.668478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:40.668440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:41.233782 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:41.233750    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkglg_40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5/node-ca/0.log"
Jul 24 15:41:41.633488 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:41.633396    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w87bk_54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab/serve-healthcheck-canary/0.log"
Jul 24 15:41:42.127265 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:42.127215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:42.127451 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:42.127303    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:42.127451 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.127413    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:41:42.127451 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.127435    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78df859bd7-tc8gh: secret "image-registry-tls" not found
Jul 24 15:41:42.127595 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.127474    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:41:42.127595 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.127491    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls podName:cae858f5-b847-4d94-9e20-224f444bc494 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:50.127475487 +0000 UTC m=+51.679715237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls") pod "image-registry-78df859bd7-tc8gh" (UID: "cae858f5-b847-4d94-9e20-224f444bc494") : secret "image-registry-tls" not found
Jul 24 15:41:42.127595 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.127521    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls podName:9a52f9e9-355e-4e1e-93ab-757fbaee85d6 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:50.127508495 +0000 UTC m=+51.679748246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls") pod "dns-default-n2cf7" (UID: "9a52f9e9-355e-4e1e-93ab-757fbaee85d6") : secret "dns-default-metrics-tls" not found
Jul 24 15:41:42.228352 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:42.228317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:42.228550 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:42.228370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:42.228550 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.228474    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:41:42.228550 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.228521    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:42.228721 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.228562    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:46.228539177 +0000 UTC m=+47.780778947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : configmap "kube-rbac-proxy" not found
Jul 24 15:41:42.228721 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:42.228583    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:46.228573684 +0000 UTC m=+47.780813437 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:43.315012 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.314969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8" event={"ID":"fd79b9a9-991e-46a5-b5b0-2cc6710106f0","Type":"ContainerStarted","Data":"2e3b71c08d2a60a83ccead6f1af30b60ab63c15374596342c307544d3c4151b7"}
Jul 24 15:41:43.315465 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.315039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8" event={"ID":"fd79b9a9-991e-46a5-b5b0-2cc6710106f0","Type":"ContainerStarted","Data":"f651cd9c407c7af0d6ff2673cc082255c136ef2014260466e7c9871873911f12"}
Jul 24 15:41:43.316430 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.316403    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz" event={"ID":"824a444c-91e2-48d9-b200-dc173ca76f82","Type":"ContainerStarted","Data":"652e6ce6e4ee03b51ec622914ea05853f7a6b1337a517678b05a69cc0922f802"}
Jul 24 15:41:43.333224 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.333173    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m7lh8" podStartSLOduration=1.914703335 podStartE2EDuration="4.33315623s" totalImagesPullingTime="2.418452895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:43.33257043 +0000 UTC m=+44.884810204" watchObservedRunningTime="2026-07-24 15:41:43.33315623 +0000 UTC m=+44.885396001"
Jul 24 15:41:43.347263 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.347199    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-t8zmz" podStartSLOduration=1.749175741 podStartE2EDuration="4.347176404s" totalImagesPullingTime="2.598000663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:43.346740018 +0000 UTC m=+44.898979790" watchObservedRunningTime="2026-07-24 15:41:43.347176404 +0000 UTC m=+44.899416177"
Jul 24 15:41:43.429224 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:43.429179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.153259 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.153217    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.260395 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.260359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.365088 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.365052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.459051 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.458965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.559640 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.559603    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:44.838250 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:44.838215    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:45.639147 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:45.639091    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:46.039694 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.039660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:46.237525 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.237458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:46.260671 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.260638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:46.260828 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.260680    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:46.260828 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:46.260788    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:41:46.260828 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:46.260803    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:46.260950 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:46.260870    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:54.260855624 +0000 UTC m=+55.813095375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:41:46.260950 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:41:46.260883    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm podName:62ba8453-57e4-4f6f-aee0-73dd75259434 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:54.260875993 +0000 UTC m=+55.813115744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkfp2" (UID: "62ba8453-57e4-4f6f-aee0-73dd75259434") : configmap "kube-rbac-proxy" not found
Jul 24 15:41:46.802241 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.802207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:46.901233 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:46.901190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:47.238725 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:47.238689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:47.434075 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:47.434031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:47.854928 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:47.854893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:47.906145 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:47.906104    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:47.984723 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:47.984686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:48.085363 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:48.085316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:48.246193 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:48.246165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:48.435287 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:48.435253    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:49.038976 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:49.038937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:49.089292 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:49.089245    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:49.240253 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:49.240218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:49.499231 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:49.499157    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:49.641093 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:49.641032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:50.192321 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.192283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:50.192719 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.192350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:50.194786 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.194766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a52f9e9-355e-4e1e-93ab-757fbaee85d6-metrics-tls\") pod \"dns-default-n2cf7\" (UID: \"9a52f9e9-355e-4e1e-93ab-757fbaee85d6\") " pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:50.194948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.194930    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"image-registry-78df859bd7-tc8gh\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") " pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:50.257299 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.257247    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:50.272566 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.272524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:50.417850 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.417818    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:41:50.420977 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:50.420949    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae858f5_b847_4d94_9e20_224f444bc494.slice/crio-31beaac093db78c736262f28a46023171d60b13f0f27f18ddedee0cabe5bd4af WatchSource:0}: Error finding container 31beaac093db78c736262f28a46023171d60b13f0f27f18ddedee0cabe5bd4af: Status 404 returned error can't find the container with id 31beaac093db78c736262f28a46023171d60b13f0f27f18ddedee0cabe5bd4af
Jul 24 15:41:50.433578 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.433544    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2cf7"]
Jul 24 15:41:50.436633 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:50.436607    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a52f9e9_355e_4e1e_93ab_757fbaee85d6.slice/crio-ca6d823482be0c643675f86e12c38f080529f0d56c5c08700c69955c0c603ec3 WatchSource:0}: Error finding container ca6d823482be0c643675f86e12c38f080529f0d56c5c08700c69955c0c603ec3: Status 404 returned error can't find the container with id ca6d823482be0c643675f86e12c38f080529f0d56c5c08700c69955c0c603ec3
Jul 24 15:41:50.448966 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.448943    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:50.629614 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.629579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:50.833776 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:50.833743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.237516 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.237474    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.341207 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.341166    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" event={"ID":"cae858f5-b847-4d94-9e20-224f444bc494","Type":"ContainerStarted","Data":"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"}
Jul 24 15:41:51.341402 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.341233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" event={"ID":"cae858f5-b847-4d94-9e20-224f444bc494","Type":"ContainerStarted","Data":"31beaac093db78c736262f28a46023171d60b13f0f27f18ddedee0cabe5bd4af"}
Jul 24 15:41:51.341402 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.341261    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:41:51.342706 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.342670    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2cf7" event={"ID":"9a52f9e9-355e-4e1e-93ab-757fbaee85d6","Type":"ContainerStarted","Data":"ca6d823482be0c643675f86e12c38f080529f0d56c5c08700c69955c0c603ec3"}
Jul 24 15:41:51.362703 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.362650    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" podStartSLOduration=22.362627331 podStartE2EDuration="22.362627331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:51.360770614 +0000 UTC m=+52.913010387" watchObservedRunningTime="2026-07-24 15:41:51.362627331 +0000 UTC m=+52.914867104"
Jul 24 15:41:51.438086 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.438050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.538206 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.538065    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.649766 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.649675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.837191 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.837083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:51.891683 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:51.891649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:52.640387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:52.640361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:53.042483 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.042450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:53.238282 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.238246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:53.350410 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.350314    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2cf7" event={"ID":"9a52f9e9-355e-4e1e-93ab-757fbaee85d6","Type":"ContainerStarted","Data":"e4786343b4fb80a8becbfb4eccea8100b5efc580a081427787cdfb9a6fe220cb"}
Jul 24 15:41:53.350410 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.350392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2cf7" event={"ID":"9a52f9e9-355e-4e1e-93ab-757fbaee85d6","Type":"ContainerStarted","Data":"a6ee12c4c8bbb104dc07419ba09440fd848cb6542fe9a9bf1003c6bd84d9b16f"}
Jul 24 15:41:53.350627 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.350430    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:41:53.367854 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.367790    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n2cf7" podStartSLOduration=17.372882468 podStartE2EDuration="19.36776954s" totalImagesPullingTime="1.994887072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:53.366565482 +0000 UTC m=+54.918805266" watchObservedRunningTime="2026-07-24 15:41:53.36776954 +0000 UTC m=+54.920009296"
Jul 24 15:41:53.429622 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.429584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:53.637164 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:53.637043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:54.331694 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.331655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:54.331694 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.331700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:54.332331 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.332305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/62ba8453-57e4-4f6f-aee0-73dd75259434-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:54.334224 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.334203    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/62ba8453-57e4-4f6f-aee0-73dd75259434-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkfp2\" (UID: \"62ba8453-57e4-4f6f-aee0-73dd75259434\") " pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:54.366903 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.366866    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkfp2"
Jul 24 15:41:54.447738 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.447691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:54.501390 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.501358    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkfp2"]
Jul 24 15:41:54.504545 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:54.504517    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ba8453_57e4_4f6f_aee0_73dd75259434.slice/crio-b18e3a56ceb8d99e9ba3b7f938ee77c8c0c3d48353048edfd6a35a54d9489990 WatchSource:0}: Error finding container b18e3a56ceb8d99e9ba3b7f938ee77c8c0c3d48353048edfd6a35a54d9489990: Status 404 returned error can't find the container with id b18e3a56ceb8d99e9ba3b7f938ee77c8c0c3d48353048edfd6a35a54d9489990
Jul 24 15:41:54.641963 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.641863    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:54.834547 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:54.834508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:55.360452 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:55.360411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkfp2" event={"ID":"62ba8453-57e4-4f6f-aee0-73dd75259434","Type":"ContainerStarted","Data":"b18e3a56ceb8d99e9ba3b7f938ee77c8c0c3d48353048edfd6a35a54d9489990"}
Jul 24 15:41:57.976453 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:57.976420    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"]
Jul 24 15:41:58.001340 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.001309    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"]
Jul 24 15:41:58.001498 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.001445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.004145 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.004101    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 15:41:58.004375 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.004116    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 15:41:58.005071 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005045    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5gn48\""
Jul 24 15:41:58.005286 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005083    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 15:41:58.005286 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005058    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 15:41:58.005538 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005517    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"]
Jul 24 15:41:58.005648 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"]
Jul 24 15:41:58.005706 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.005668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.008279 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.008254    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 15:41:58.008616 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.008589    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 15:41:58.009407 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.009215    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 15:41:58.010153 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.009828    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 15:41:58.011968 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.011172    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:41:58.045311 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.045274    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-85b877574b-4k6sp"]
Jul 24 15:41:58.050532 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.050504    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.060755 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.060724    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85b877574b-4k6sp"]
Jul 24 15:41:58.064071 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-tls\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064276 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ddca80d-ac16-4302-a00b-1449186e6c96-ca-trust-extracted\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064276 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd4d2\" (UniqueName: \"kubernetes.io/projected/891dd825-1dee-4fc1-bc6b-a4036b9349c0-kube-api-access-zd4d2\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064276 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-bound-sa-token\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064276 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064295    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064330    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86e3ade2-6792-495b-a028-fa921b34645d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-image-registry-private-configuration\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-certificates\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064414    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/891dd825-1dee-4fc1-bc6b-a4036b9349c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8gms\" (UniqueName: \"kubernetes.io/projected/86e3ade2-6792-495b-a028-fa921b34645d-kube-api-access-t8gms\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.064476 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064462    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064805 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-trusted-ca\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064805 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jph9v\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-kube-api-access-jph9v\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.064805 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.064805 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.064588    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-installation-pull-secrets\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.066348 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.066317    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:58.143382 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.143348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:58.164938 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.164903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.164938 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.164945    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86e3ade2-6792-495b-a028-fa921b34645d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.164973    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-image-registry-private-configuration\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-certificates\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/891dd825-1dee-4fc1-bc6b-a4036b9349c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8gms\" (UniqueName: \"kubernetes.io/projected/86e3ade2-6792-495b-a028-fa921b34645d-kube-api-access-t8gms\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-trusted-ca\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165156    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jph9v\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-kube-api-access-jph9v\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-installation-pull-secrets\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-tls\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ddca80d-ac16-4302-a00b-1449186e6c96-ca-trust-extracted\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd4d2\" (UniqueName: \"kubernetes.io/projected/891dd825-1dee-4fc1-bc6b-a4036b9349c0-kube-api-access-zd4d2\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.165948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165371    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-bound-sa-token\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.165948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.165403    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.166763 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.166731    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-certificates\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.167519 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.167491    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/891dd825-1dee-4fc1-bc6b-a4036b9349c0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.168076 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.168047    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ddca80d-ac16-4302-a00b-1449186e6c96-ca-trust-extracted\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.168223 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.168144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ddca80d-ac16-4302-a00b-1449186e6c96-trusted-ca\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.169050 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.169018    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-image-registry-private-configuration\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.170616 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.170590    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.170971 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.170953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.171029 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.171012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ddca80d-ac16-4302-a00b-1449186e6c96-installation-pull-secrets\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.171798 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.171759    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-registry-tls\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.171798 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.171793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-ca\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.171959 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.171940    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86e3ade2-6792-495b-a028-fa921b34645d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.173791 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.173770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/891dd825-1dee-4fc1-bc6b-a4036b9349c0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.177186 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.177110    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd4d2\" (UniqueName: \"kubernetes.io/projected/891dd825-1dee-4fc1-bc6b-a4036b9349c0-kube-api-access-zd4d2\") pod \"cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj\" (UID: \"891dd825-1dee-4fc1-bc6b-a4036b9349c0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.177574 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.177552    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jph9v\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-kube-api-access-jph9v\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.177887 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.177860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ddca80d-ac16-4302-a00b-1449186e6c96-bound-sa-token\") pod \"image-registry-85b877574b-4k6sp\" (UID: \"4ddca80d-ac16-4302-a00b-1449186e6c96\") " pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.178367 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.178346    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8gms\" (UniqueName: \"kubernetes.io/projected/86e3ade2-6792-495b-a028-fa921b34645d-kube-api-access-t8gms\") pod \"managed-serviceaccount-addon-agent-fcdffb896-g49d7\" (UID: \"86e3ade2-6792-495b-a028-fa921b34645d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.242509 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.242468    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:58.323749 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.323720    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"
Jul 24 15:41:58.331910 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.331869    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"
Jul 24 15:41:58.357492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.357340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:58.372747 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.372719    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:41:58.420208 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.420172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:58.489710 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:58.489673    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:41:59.948459 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:41:59.948388    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj"]
Jul 24 15:41:59.952662 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:41:59.952628    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod891dd825_1dee_4fc1_bc6b_a4036b9349c0.slice/crio-369c61bfee296dfa7b8adb21e66ef3220acd18c4affc278452d13e165ef7e6ed WatchSource:0}: Error finding container 369c61bfee296dfa7b8adb21e66ef3220acd18c4affc278452d13e165ef7e6ed: Status 404 returned error can't find the container with id 369c61bfee296dfa7b8adb21e66ef3220acd18c4affc278452d13e165ef7e6ed
Jul 24 15:42:00.170059 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.169976    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7"]
Jul 24 15:42:00.172975 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:00.172946    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86e3ade2_6792_495b_a028_fa921b34645d.slice/crio-c2be5b0f6258651dac77383dcf7a846757800d6377bb359bee4705eb1856c301 WatchSource:0}: Error finding container c2be5b0f6258651dac77383dcf7a846757800d6377bb359bee4705eb1856c301: Status 404 returned error can't find the container with id c2be5b0f6258651dac77383dcf7a846757800d6377bb359bee4705eb1856c301
Jul 24 15:42:00.175544 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.175475    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-85b877574b-4k6sp"]
Jul 24 15:42:00.178923 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:00.178891    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ddca80d_ac16_4302_a00b_1449186e6c96.slice/crio-a290b725a5b77bf3974a5cb4e322a348b3f6256924dbbc3924d5929d3db83c51 WatchSource:0}: Error finding container a290b725a5b77bf3974a5cb4e322a348b3f6256924dbbc3924d5929d3db83c51: Status 404 returned error can't find the container with id a290b725a5b77bf3974a5cb4e322a348b3f6256924dbbc3924d5929d3db83c51
Jul 24 15:42:00.274254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.274226    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-89rdl"
Jul 24 15:42:00.380360 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.380320    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b877574b-4k6sp" event={"ID":"4ddca80d-ac16-4302-a00b-1449186e6c96","Type":"ContainerStarted","Data":"4168bd7500056702d2ac71b7da8d7dcb7d50bc2564766503e3903087930d3a73"}
Jul 24 15:42:00.380561 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.380384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-85b877574b-4k6sp" event={"ID":"4ddca80d-ac16-4302-a00b-1449186e6c96","Type":"ContainerStarted","Data":"a290b725a5b77bf3974a5cb4e322a348b3f6256924dbbc3924d5929d3db83c51"}
Jul 24 15:42:00.380561 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.380449    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:42:00.381579 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.381545    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7" event={"ID":"86e3ade2-6792-495b-a028-fa921b34645d","Type":"ContainerStarted","Data":"c2be5b0f6258651dac77383dcf7a846757800d6377bb359bee4705eb1856c301"}
Jul 24 15:42:00.383000 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.382929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj" event={"ID":"891dd825-1dee-4fc1-bc6b-a4036b9349c0","Type":"ContainerStarted","Data":"369c61bfee296dfa7b8adb21e66ef3220acd18c4affc278452d13e165ef7e6ed"}
Jul 24 15:42:00.384581 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.384561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkfp2" event={"ID":"62ba8453-57e4-4f6f-aee0-73dd75259434","Type":"ContainerStarted","Data":"2a3fa7bf29c36252ddde86b9d97c01380376e31c1a039449e800a7b5efc3d5b1"}
Jul 24 15:42:00.399725 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.399664    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-85b877574b-4k6sp" podStartSLOduration=2.39964193 podStartE2EDuration="2.39964193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:00.397972731 +0000 UTC m=+61.950212505" watchObservedRunningTime="2026-07-24 15:42:00.39964193 +0000 UTC m=+61.951881705"
Jul 24 15:42:00.402376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.402348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:00.462836 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:00.462748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:03.356973 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.356942    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n2cf7"
Jul 24 15:42:03.719777 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.719691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:42:03.719947 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.719780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:42:03.722497 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.722467    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:42:03.722661 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.722588    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:42:03.733509 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.733470    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44fcb14e-69e3-4b41-8baa-ccb7b8f4793d-original-pull-secret\") pod \"global-pull-secret-syncer-59zdt\" (UID: \"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d\") " pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:42:03.733690 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.733641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afd27a-11a7-4701-9b08-408dcc75cc08-metrics-certs\") pod \"network-metrics-daemon-jbfbj\" (UID: \"f7afd27a-11a7-4701-9b08-408dcc75cc08\") " pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:42:03.758328 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.758297    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nh5k9\""
Jul 24 15:42:03.766463 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.766424    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jbfbj"
Jul 24 15:42:03.820266 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.820212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:42:03.823376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.823345    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:42:03.833881 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.833848    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:42:03.844995 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.844958    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh4p9\" (UniqueName: \"kubernetes.io/projected/3eb29b7c-fcff-4c45-9561-1bf24918177b-kube-api-access-zh4p9\") pod \"network-check-target-44n2k\" (UID: \"3eb29b7c-fcff-4c45-9561-1bf24918177b\") " pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:42:03.852389 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:03.852355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-59zdt"
Jul 24 15:42:04.064695 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:04.064656    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvmsc\""
Jul 24 15:42:04.072826 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:04.072785    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:42:06.839664 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:06.839622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:06.967018 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:06.966939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:07.112114 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.111684    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jbfbj"]
Jul 24 15:42:07.122506 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:07.122335    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7afd27a_11a7_4701_9b08_408dcc75cc08.slice/crio-11ae2925e31623deb8e244c617dcaef6cf15c1c3eb0fa0d89a3a54fd3715fcd0 WatchSource:0}: Error finding container 11ae2925e31623deb8e244c617dcaef6cf15c1c3eb0fa0d89a3a54fd3715fcd0: Status 404 returned error can't find the container with id 11ae2925e31623deb8e244c617dcaef6cf15c1c3eb0fa0d89a3a54fd3715fcd0
Jul 24 15:42:07.125089 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.125057    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-59zdt"]
Jul 24 15:42:07.129408 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:07.129369    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44fcb14e_69e3_4b41_8baa_ccb7b8f4793d.slice/crio-27295b29270158aa37f0f48327cad8191d351b16437d000d515e3f5d30eb5612 WatchSource:0}: Error finding container 27295b29270158aa37f0f48327cad8191d351b16437d000d515e3f5d30eb5612: Status 404 returned error can't find the container with id 27295b29270158aa37f0f48327cad8191d351b16437d000d515e3f5d30eb5612
Jul 24 15:42:07.129933 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.129878    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-44n2k"]
Jul 24 15:42:07.132031 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:07.132006    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb29b7c_fcff_4c45_9561_1bf24918177b.slice/crio-cb30abae9b40fe64e1c7a85f5a65504c4c75a1ab4aa30516ec86c0e1eee480b4 WatchSource:0}: Error finding container cb30abae9b40fe64e1c7a85f5a65504c4c75a1ab4aa30516ec86c0e1eee480b4: Status 404 returned error can't find the container with id cb30abae9b40fe64e1c7a85f5a65504c4c75a1ab4aa30516ec86c0e1eee480b4
Jul 24 15:42:07.426072 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.425972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkfp2" event={"ID":"62ba8453-57e4-4f6f-aee0-73dd75259434","Type":"ContainerStarted","Data":"12afaeb9aa4fd6ef5fcf362fcff75aa8d8bcce9659ee7e1be1258f2ba6f53f5f"}
Jul 24 15:42:07.427150 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.427100    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-59zdt" event={"ID":"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d","Type":"ContainerStarted","Data":"27295b29270158aa37f0f48327cad8191d351b16437d000d515e3f5d30eb5612"}
Jul 24 15:42:07.428345 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.428312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jbfbj" event={"ID":"f7afd27a-11a7-4701-9b08-408dcc75cc08","Type":"ContainerStarted","Data":"11ae2925e31623deb8e244c617dcaef6cf15c1c3eb0fa0d89a3a54fd3715fcd0"}
Jul 24 15:42:07.429482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.429458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-44n2k" event={"ID":"3eb29b7c-fcff-4c45-9561-1bf24918177b","Type":"ContainerStarted","Data":"cb30abae9b40fe64e1c7a85f5a65504c4c75a1ab4aa30516ec86c0e1eee480b4"}
Jul 24 15:42:07.430934 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.430902    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7" event={"ID":"86e3ade2-6792-495b-a028-fa921b34645d","Type":"ContainerStarted","Data":"43c254ebfd0012ed39312a180fca3bbd53630a07bf2ae8d63942efebaae9e1c9"}
Jul 24 15:42:07.432913 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.432892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj" event={"ID":"891dd825-1dee-4fc1-bc6b-a4036b9349c0","Type":"ContainerStarted","Data":"3ed4892a1f0f0e0f0feb55cc2fcaecc89140549fc2bbb6f4b4842e45028ff0c9"}
Jul 24 15:42:07.433019 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.432922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj" event={"ID":"891dd825-1dee-4fc1-bc6b-a4036b9349c0","Type":"ContainerStarted","Data":"4643ef381e2338b28afaf160c521c434c28a59f6b3ff0c6e83ec1b6a9dc6a7d8"}
Jul 24 15:42:07.433019 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.432941    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj" event={"ID":"891dd825-1dee-4fc1-bc6b-a4036b9349c0","Type":"ContainerStarted","Data":"933ca2d360644d804c9d70036df97f1db005ec6e96c1fbb1510f17abfcd5c3aa"}
Jul 24 15:42:07.447646 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.447580    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fcdffb896-g49d7" podStartSLOduration=3.707665716 podStartE2EDuration="10.447560843s" totalImagesPullingTime="6.739895127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:07.44683299 +0000 UTC m=+68.999072763" watchObservedRunningTime="2026-07-24 15:42:07.447560843 +0000 UTC m=+68.999800593"
Jul 24 15:42:07.467553 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:07.467485    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f6c6bcb7d-sz6jj" podStartSLOduration=3.458794801 podStartE2EDuration="10.4674614s" totalImagesPullingTime="7.008666599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:07.466703548 +0000 UTC m=+69.018943323" watchObservedRunningTime="2026-07-24 15:42:07.4674614 +0000 UTC m=+69.019701174"
Jul 24 15:42:08.019537 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:08.019194    2567 patch_prober.go:28] interesting pod/image-registry-78df859bd7-tc8gh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 15:42:08.019537 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:08.019270    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" podUID="cae858f5-b847-4d94-9e20-224f444bc494" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:42:08.374773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:08.374606    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:08.473753 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:08.473697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:08.527024 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:08.526145    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:09.449174 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:09.448531    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jbfbj" event={"ID":"f7afd27a-11a7-4701-9b08-408dcc75cc08","Type":"ContainerStarted","Data":"b02613d44f1bc3f4aecf468177da82dbc672a9c472afabc6ab5a6f3a20913476"}
Jul 24 15:42:09.449174 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:09.448605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jbfbj" event={"ID":"f7afd27a-11a7-4701-9b08-408dcc75cc08","Type":"ContainerStarted","Data":"88ec7562d9802548912142544a027f34b2fe19dedfcabf851228b88ae6e77176"}
Jul 24 15:42:09.547591 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:09.547532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:09.649342 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:09.649297    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.007818 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.007780    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.104646 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.104610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.175833 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.175793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.230614 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.230568    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.484874 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.484841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.582521 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.582465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:11.637879 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:11.637846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.079282 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.079190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.193667 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.193635    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.294539 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.294492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.395145 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.395043    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.469790 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.469752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkfp2" event={"ID":"62ba8453-57e4-4f6f-aee0-73dd75259434","Type":"ContainerStarted","Data":"9f962b034b658f79df01a7e5406104d7fc2c5e705c563900a5c8f352dde2f87a"}
Jul 24 15:42:14.471322 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.471288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-59zdt" event={"ID":"44fcb14e-69e3-4b41-8baa-ccb7b8f4793d","Type":"ContainerStarted","Data":"4d5505151e03b330623f8c98ef773f98c25f24d6e2468ffc064ee95c5c879349"}
Jul 24 15:42:14.472737 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.472715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-44n2k" event={"ID":"3eb29b7c-fcff-4c45-9561-1bf24918177b","Type":"ContainerStarted","Data":"ea6afd23081d11a7cacc286504ea16e4ddc654650a51ced94f06a9c912db67d6"}
Jul 24 15:42:14.472855 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.472844    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:42:14.487482 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.487368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jbfbj" podStartSLOduration=74.201652338 podStartE2EDuration="1m15.487346269s" totalImagesPullingTime="1.285693931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:09.465737598 +0000 UTC m=+71.017977371" watchObservedRunningTime="2026-07-24 15:42:14.487346269 +0000 UTC m=+76.039586044"
Jul 24 15:42:14.487994 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.487959    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fkfp2" podStartSLOduration=17.385294848 podStartE2EDuration="36.487950382s" totalImagesPullingTime="19.102655534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:41:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:14.486380917 +0000 UTC m=+76.038620692" watchObservedRunningTime="2026-07-24 15:42:14.487950382 +0000 UTC m=+76.040190174"
Jul 24 15:42:14.491234 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.491205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.502577 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.502520    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-59zdt" podStartSLOduration=68.828407952 podStartE2EDuration="1m15.502501922s" totalImagesPullingTime="6.67409397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:14.501850855 +0000 UTC m=+76.054090630" watchObservedRunningTime="2026-07-24 15:42:14.502501922 +0000 UTC m=+76.054741717"
Jul 24 15:42:14.520455 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.520400    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-44n2k" podStartSLOduration=68.859151841 podStartE2EDuration="1m15.520383824s" totalImagesPullingTime="6.661231983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:14.520063694 +0000 UTC m=+76.072303482" watchObservedRunningTime="2026-07-24 15:42:14.520383824 +0000 UTC m=+76.072623597"
Jul 24 15:42:14.843918 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.843880    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:14.903248 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:14.903211    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.216166 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.216068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.288694 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.288625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.376013 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.375975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.476681 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.476586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.526281 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.526255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.653439 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.653401    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.792177 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.792115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:15.893477 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:15.893443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:16.062068 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:16.061973    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:17.273339 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:17.273310    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:17.374902 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:17.374871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:17.475147 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:17.475085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:17.575029 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:17.574942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:18.016353 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:18.016322    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:42:21.404889 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.404751    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-85b877574b-4k6sp"
Jul 24 15:42:21.448241 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.448188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:21.545835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.545764    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:21.661330 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.661233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:21.762723 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.762689    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:21.863369 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.863337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:21.964320 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:21.964231    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.034255 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.034215    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" podUID="cae858f5-b847-4d94-9e20-224f444bc494" containerName="registry" containerID="cri-o://c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d" gracePeriod=30
Jul 24 15:42:23.151488 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.151461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.252257 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.252216    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.283440 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.283413    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:42:23.355918 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.355820    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.378768 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.378733    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.378915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.378778    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.378915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.378821    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.378915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.378857    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.378915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.378892    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.379196 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.379007    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5m6\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.379196 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.379058    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.379196 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.379102    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token\") pod \"cae858f5-b847-4d94-9e20-224f444bc494\" (UID: \"cae858f5-b847-4d94-9e20-224f444bc494\") "
Jul 24 15:42:23.379513 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.379488    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:42:23.379739 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.379681    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:42:23.381685 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.381623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:42:23.382021 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.381974    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:42:23.382143 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.382044    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:42:23.382215 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.382117    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:42:23.382318 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.382299    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6" (OuterVolumeSpecName: "kube-api-access-dd5m6") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "kube-api-access-dd5m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:42:23.388010 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.387982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cae858f5-b847-4d94-9e20-224f444bc494" (UID: "cae858f5-b847-4d94-9e20-224f444bc494"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:42:23.479708 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479665    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-registry-certificates\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479708 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479696    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cae858f5-b847-4d94-9e20-224f444bc494-trusted-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479708 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479709    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-image-registry-private-configuration\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479708 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479718    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5m6\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-kube-api-access-dd5m6\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479962 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479727    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-registry-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479962 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479735    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cae858f5-b847-4d94-9e20-224f444bc494-bound-sa-token\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479962 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479743    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cae858f5-b847-4d94-9e20-224f444bc494-ca-trust-extracted\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.479962 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.479751    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cae858f5-b847-4d94-9e20-224f444bc494-installation-pull-secrets\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:42:23.497113 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.497070    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.506302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.506255    2567 generic.go:356] "Generic (PLEG): container finished" podID="cae858f5-b847-4d94-9e20-224f444bc494" containerID="c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d" exitCode=0
Jul 24 15:42:23.506302 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.506301    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh"
Jul 24 15:42:23.506508 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.506334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" event={"ID":"cae858f5-b847-4d94-9e20-224f444bc494","Type":"ContainerDied","Data":"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"}
Jul 24 15:42:23.506508 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.506392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78df859bd7-tc8gh" event={"ID":"cae858f5-b847-4d94-9e20-224f444bc494","Type":"ContainerDied","Data":"31beaac093db78c736262f28a46023171d60b13f0f27f18ddedee0cabe5bd4af"}
Jul 24 15:42:23.506508 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.506418    2567 scope.go:117] "RemoveContainer" containerID="c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"
Jul 24 15:42:23.517769 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.517748    2567 scope.go:117] "RemoveContainer" containerID="c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"
Jul 24 15:42:23.518083 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:42:23.518064    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d\": container with ID starting with c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d not found: ID does not exist" containerID="c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"
Jul 24 15:42:23.518277 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.518091    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d"} err="failed to get container status \"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d\": rpc error: code = NotFound desc = could not find container \"c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d\": container with ID starting with c77723a1b2fbe1ba51614a3b1f49a819ee43a7d13c16d7c25d2497457c86f40d not found: ID does not exist"
Jul 24 15:42:23.536574 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.536538    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:42:23.541754 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.541724    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78df859bd7-tc8gh"]
Jul 24 15:42:23.585991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.585897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.661402 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.661316    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l4lls"]
Jul 24 15:42:23.661663 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.661650    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cae858f5-b847-4d94-9e20-224f444bc494" containerName="registry"
Jul 24 15:42:23.661709 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.661668    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cae858f5-b847-4d94-9e20-224f444bc494" containerName="registry"
Jul 24 15:42:23.666154 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.662105    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cae858f5-b847-4d94-9e20-224f444bc494" containerName="registry"
Jul 24 15:42:23.669236 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.669193    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.672025 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.671998    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 15:42:23.672189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.672068    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 15:42:23.672189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.672089    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 15:42:23.672189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.671999    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 15:42:23.672360 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.672318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 15:42:23.672521 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.672498    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bk4xx\""
Jul 24 15:42:23.672637 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.672527    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 15:42:23.782218 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-sys\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782218 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdr2q\" (UniqueName: \"kubernetes.io/projected/c54c4da7-de32-4563-8729-ef83c39b3fb5-kube-api-access-zdr2q\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782357    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-tls\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782428    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-root\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-wtmp\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782460    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-metrics-client-ca\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.782487 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.782478    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-textfile\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.848106 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.848067    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.883780 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883744    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883793    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-tls\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883812    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-root\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-wtmp\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-metrics-client-ca\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-root\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-textfile\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.883980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883954    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-sys\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.883982    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdr2q\" (UniqueName: \"kubernetes.io/projected/c54c4da7-de32-4563-8729-ef83c39b3fb5-kube-api-access-zdr2q\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884041    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-wtmp\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c54c4da7-de32-4563-8729-ef83c39b3fb5-sys\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-textfile\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884688 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884671    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.884796 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.884776    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54c4da7-de32-4563-8729-ef83c39b3fb5-metrics-client-ca\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.886283 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.886259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-tls\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.886403 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.886376    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c54c4da7-de32-4563-8729-ef83c39b3fb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.894398 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.894372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdr2q\" (UniqueName: \"kubernetes.io/projected/c54c4da7-de32-4563-8729-ef83c39b3fb5-kube-api-access-zdr2q\") pod \"node-exporter-l4lls\" (UID: \"c54c4da7-de32-4563-8729-ef83c39b3fb5\") " pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.948534 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.948411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:23.980172 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:23.980114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l4lls"
Jul 24 15:42:23.992313 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:42:23.992282    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54c4da7_de32_4563_8729_ef83c39b3fb5.slice/crio-48b126dc1ec5462c24b9e5b54090eacfc8b7e3ce7642fc07f2c3fb68bbb04a7d WatchSource:0}: Error finding container 48b126dc1ec5462c24b9e5b54090eacfc8b7e3ce7642fc07f2c3fb68bbb04a7d: Status 404 returned error can't find the container with id 48b126dc1ec5462c24b9e5b54090eacfc8b7e3ce7642fc07f2c3fb68bbb04a7d
Jul 24 15:42:24.049305 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.049207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:24.191699 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.191667    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:24.511557 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.511521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4lls" event={"ID":"c54c4da7-de32-4563-8729-ef83c39b3fb5","Type":"ContainerStarted","Data":"48b126dc1ec5462c24b9e5b54090eacfc8b7e3ce7642fc07f2c3fb68bbb04a7d"}
Jul 24 15:42:24.573851 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.573816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:24.656083 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.656048    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:24.724471 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.724432    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:24.948568 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:24.948481    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cae858f5-b847-4d94-9e20-224f444bc494" path="/var/lib/kubelet/pods/cae858f5-b847-4d94-9e20-224f444bc494/volumes"
Jul 24 15:42:25.122838 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:25.122803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:25.518237 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:25.518204    2567 generic.go:356] "Generic (PLEG): container finished" podID="c54c4da7-de32-4563-8729-ef83c39b3fb5" containerID="74e40d72d8a06480d7f82da9701198f51890a8023dc91a17226aa4f5d04ba141" exitCode=0
Jul 24 15:42:25.518424 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:25.518278    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4lls" event={"ID":"c54c4da7-de32-4563-8729-ef83c39b3fb5","Type":"ContainerDied","Data":"74e40d72d8a06480d7f82da9701198f51890a8023dc91a17226aa4f5d04ba141"}
Jul 24 15:42:26.520897 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.520857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:26.524449 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.524411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4lls" event={"ID":"c54c4da7-de32-4563-8729-ef83c39b3fb5","Type":"ContainerStarted","Data":"c64ed78efd547155dcbf079d67e396b8b853e53292f77cc8b4a2d01e5693ab4e"}
Jul 24 15:42:26.524595 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.524468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l4lls" event={"ID":"c54c4da7-de32-4563-8729-ef83c39b3fb5","Type":"ContainerStarted","Data":"0b5c7188d3e2deeaa1b7ecc90b7fa3351932fbf6d950a5dcbda112335315451c"}
Jul 24 15:42:26.544915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.544861    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l4lls" podStartSLOduration=2.508669814 podStartE2EDuration="3.544845747s" totalImagesPullingTime="961.398585ms" totalInitContainerRuntime="74.777348ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:26.543160427 +0000 UTC m=+88.095400202" watchObservedRunningTime="2026-07-24 15:42:26.544845747 +0000 UTC m=+88.097085520"
Jul 24 15:42:26.607085 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.607045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:26.671314 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.671275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:26.774935 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:26.774843    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:28.348775 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:28.348737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:28.416506 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:28.416475    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:28.673037 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:28.672954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:29.489319 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:29.489279    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:29.819265 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:29.819153    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:29.882776 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:29.882726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:29.942311 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:29.942258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:33.674773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:33.674737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:33.732528 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:33.732491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:37.120095 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:37.120060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:37.173861 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:37.173822    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:41.489329 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:41.489290    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:41.546486 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:41.546450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:42.519335 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:42.519300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:42.622671 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:42.622633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:44.302256 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:44.302220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:44.641422 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:44.641338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:44.692082 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:44.692039    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:45.479866 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:45.479834    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-44n2k"
Jul 24 15:42:51.839533 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:51.839496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:51.891884 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:51.891844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:52.746428 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:52.746387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:52.847379 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:52.847344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:52.907922 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:52.907874    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:59.198929 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:59.198841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:59.300366 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:59.300326    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:42:59.400840 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:42:59.400792    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:01.388876 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:01.388834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:06.840849 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:06.840811    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:06.897506 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:06.897472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:42.777274 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:42.777239    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:42.877490 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:42.877455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:51.706093 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:51.706004    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:51.761017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:51.760967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:51.849603 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:51.849569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:51.947976 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:51.947933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:52.115190 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:52.115149    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:43:52.170945 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:43:52.170896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:03.058917 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:03.058877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:03.111549 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:03.111518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:03.174001 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:03.173959    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:56.375540 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:56.375502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:58.001949 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:58.001910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:44:58.102119 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:44:58.102087    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:07.070604 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:07.070558    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:07.117864 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:07.117827    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:09.946557 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:09.946511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:10.048236 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:10.048194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:18.342668 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:18.342627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:18.441291 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:18.441249    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:18.492442 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:18.492409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:21.842869 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:21.842778    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:21.943714 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:21.943672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:22.110360 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:22.110275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:27.162200 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:27.162159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:27.263205 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:27.263167    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:32.407859 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:32.407823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:41.336850 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:41.336809    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:41.430971 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:41.430935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:45:58.791886 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:45:58.791856    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 15:46:08.046509 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:08.046469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:09.343184 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:09.343150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:11.663024 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:11.662984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:11.763257 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:11.763220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:17.068861 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:17.068830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:17.124585 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:17.124546    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:25.981002 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:25.980948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:26.106772 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:26.106741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:26.222876 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:26.222831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:28.501408 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:28.501370    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:28.709943 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:28.709910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:36.115588 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:36.115553    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:36.940191 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:36.940120    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:37.040198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:37.040162    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:37.625810 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:37.625774    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:37.677672 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:37.677643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:38.681443 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.681336    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:38.781301 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.781265    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"]
Jul 24 15:46:38.784181 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.784164    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.786649 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.786631    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 15:46:38.786770 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.786663    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hd42t\""
Jul 24 15:46:38.787576 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.787556    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:46:38.799433 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.798091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"]
Jul 24 15:46:38.829638 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.829598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9d192d-5116-4d6b-bc2a-125bef5dd963-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.829813 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.829656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9bb4\" (UniqueName: \"kubernetes.io/projected/1d9d192d-5116-4d6b-bc2a-125bef5dd963-kube-api-access-m9bb4\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.931026 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.930984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9d192d-5116-4d6b-bc2a-125bef5dd963-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.931232 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.931067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9bb4\" (UniqueName: \"kubernetes.io/projected/1d9d192d-5116-4d6b-bc2a-125bef5dd963-kube-api-access-m9bb4\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.931399 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.931378    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9d192d-5116-4d6b-bc2a-125bef5dd963-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:38.940426 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:38.940363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9bb4\" (UniqueName: \"kubernetes.io/projected/1d9d192d-5116-4d6b-bc2a-125bef5dd963-kube-api-access-m9bb4\") pod \"openshift-lws-operator-fd8ccff4c-ztp9q\" (UID: \"1d9d192d-5116-4d6b-bc2a-125bef5dd963\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:39.093092 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:39.093036    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"
Jul 24 15:46:39.229343 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:39.229317    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q"]
Jul 24 15:46:39.232118 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:46:39.232092    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d9d192d_5116_4d6b_bc2a_125bef5dd963.slice/crio-08d7581d065076ce64d6e02ccf3a54f8623f253e334641fcd2e129bcc910408b WatchSource:0}: Error finding container 08d7581d065076ce64d6e02ccf3a54f8623f253e334641fcd2e129bcc910408b: Status 404 returned error can't find the container with id 08d7581d065076ce64d6e02ccf3a54f8623f253e334641fcd2e129bcc910408b
Jul 24 15:46:39.233615 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:39.233598    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:46:39.371331 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:39.371298    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q" event={"ID":"1d9d192d-5116-4d6b-bc2a-125bef5dd963","Type":"ContainerStarted","Data":"08d7581d065076ce64d6e02ccf3a54f8623f253e334641fcd2e129bcc910408b"}
Jul 24 15:46:42.385489 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:42.385449    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q" event={"ID":"1d9d192d-5116-4d6b-bc2a-125bef5dd963","Type":"ContainerStarted","Data":"fd2eae1619679aff7a1d7ade25c10b2fc5abad2ef3392a6963a5f65024520124"}
Jul 24 15:46:42.401424 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:42.401373    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ztp9q" podStartSLOduration=1.940884238 podStartE2EDuration="4.401357381s" totalImagesPullingTime="2.460473143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:42.400642655 +0000 UTC m=+343.952882429" watchObservedRunningTime="2026-07-24 15:46:42.401357381 +0000 UTC m=+343.953597155"
Jul 24 15:46:43.169274 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:43.169241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:43.270295 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:43.270244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:44.874319 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:44.874287    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:58.130463 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:58.130356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:58.222385 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:58.222342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:58.714900 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:58.714861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:58.990266 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:58.990225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.089684 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.089642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.141604 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.141574    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"]
Jul 24 15:46:59.147652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.147628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.149885 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.149860    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 15:46:59.150457 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.150439    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 15:46:59.150570 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.150456    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rkv9t\""
Jul 24 15:46:59.150570 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.150496    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 15:46:59.150787 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.150768    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 15:46:59.170489 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.170454    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"]
Jul 24 15:46:59.268654 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.268561    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.280471 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.280437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlb6m\" (UniqueName: \"kubernetes.io/projected/ec6adce5-79bb-445a-bda5-69cbbe11b0db-kube-api-access-tlb6m\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.280612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.280484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-tmp\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.280612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.280548    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-manifests\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.280612 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.280579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.280717 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.280617    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-webhook-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.369963 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.369925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.381837 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.381808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-tmp\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.381991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.381854    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-manifests\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.381991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.381876    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.381991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.381899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-webhook-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.381991 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.381925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlb6m\" (UniqueName: \"kubernetes.io/projected/ec6adce5-79bb-445a-bda5-69cbbe11b0db-kube-api-access-tlb6m\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.382347 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.382316    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-manifests\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.384391 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.384360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec6adce5-79bb-445a-bda5-69cbbe11b0db-tmp\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.384594 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.384574    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-webhook-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.384705 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.384684    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec6adce5-79bb-445a-bda5-69cbbe11b0db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.394013 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.393981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlb6m\" (UniqueName: \"kubernetes.io/projected/ec6adce5-79bb-445a-bda5-69cbbe11b0db-kube-api-access-tlb6m\") pod \"opendatahub-operator-controller-manager-6798866cc5-hmnrs\" (UID: \"ec6adce5-79bb-445a-bda5-69cbbe11b0db\") " pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.458327 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.458289    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:46:59.469841 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.469805    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.577963 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.577932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.603032 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.603004    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"]
Jul 24 15:46:59.605746 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:46:59.605702    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6adce5_79bb_445a_bda5_69cbbe11b0db.slice/crio-a134e1b49c60a1390812e7a0bdae2e7047a6648d3dd2333f9d6ae6a7b4654988 WatchSource:0}: Error finding container a134e1b49c60a1390812e7a0bdae2e7047a6648d3dd2333f9d6ae6a7b4654988: Status 404 returned error can't find the container with id a134e1b49c60a1390812e7a0bdae2e7047a6648d3dd2333f9d6ae6a7b4654988
Jul 24 15:46:59.669514 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.669477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.772242 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.772210    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:46:59.871082 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:46:59.870984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:00.453807 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:00.453737    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs" event={"ID":"ec6adce5-79bb-445a-bda5-69cbbe11b0db","Type":"ContainerStarted","Data":"a134e1b49c60a1390812e7a0bdae2e7047a6648d3dd2333f9d6ae6a7b4654988"}
Jul 24 15:47:03.468970 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:03.468929    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs" event={"ID":"ec6adce5-79bb-445a-bda5-69cbbe11b0db","Type":"ContainerStarted","Data":"99a17ba87ac5342406e0d50ec83071cbf6f84ac1fe32e57e2cd2a8a7d845da44"}
Jul 24 15:47:04.474632 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:04.474597    2567 generic.go:356] "Generic (PLEG): container finished" podID="ec6adce5-79bb-445a-bda5-69cbbe11b0db" containerID="99a17ba87ac5342406e0d50ec83071cbf6f84ac1fe32e57e2cd2a8a7d845da44" exitCode=0
Jul 24 15:47:04.475037 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:04.474682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs" event={"ID":"ec6adce5-79bb-445a-bda5-69cbbe11b0db","Type":"ContainerDied","Data":"99a17ba87ac5342406e0d50ec83071cbf6f84ac1fe32e57e2cd2a8a7d845da44"}
Jul 24 15:47:05.480723 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:05.480687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs" event={"ID":"ec6adce5-79bb-445a-bda5-69cbbe11b0db","Type":"ContainerStarted","Data":"b745eb88a2eaf57030bc42e86ee4ac69b2c64432ec08ca4d1abeb82f66ff3176"}
Jul 24 15:47:05.481119 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:05.480793    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:47:05.502408 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:05.502349    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs" podStartSLOduration=2.021433081 podStartE2EDuration="6.502331063s" totalImagesPullingTime="3.481378726s" totalInitContainerRuntime="999.519256ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:05.50073265 +0000 UTC m=+367.052972823" watchObservedRunningTime="2026-07-24 15:47:05.502331063 +0000 UTC m=+367.054570837"
Jul 24 15:47:11.197270 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:11.197227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:11.297704 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:11.297668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:16.487038 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:16.487011    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6798866cc5-hmnrs"
Jul 24 15:47:18.030681 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.030643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:18.141667 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.141632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:18.789900 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.789867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:18.896600 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.896571    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5688cff95c-j5665"]
Jul 24 15:47:18.905034 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.904996    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:18.908765 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.908732    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 15:47:18.908765 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.908742    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6m6g4\""
Jul 24 15:47:18.908765 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.908759    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 15:47:18.909103 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.908803    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 15:47:18.909103 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.908748    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 15:47:18.909423 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.909394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:18.917616 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.917585    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5688cff95c-j5665"]
Jul 24 15:47:18.997647 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:18.997598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:19.046475 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.046369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tls-certs\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.046475 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.046452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnvbq\" (UniqueName: \"kubernetes.io/projected/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-kube-api-access-jnvbq\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.046896 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.046494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tmp\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.094744 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.094709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:19.147950 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.147906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tls-certs\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.148198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.147997    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnvbq\" (UniqueName: \"kubernetes.io/projected/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-kube-api-access-jnvbq\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.148198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.148032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tmp\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.150534 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.150502    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tmp\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.150761 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.150743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-tls-certs\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.161442 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.161403    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnvbq\" (UniqueName: \"kubernetes.io/projected/1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f-kube-api-access-jnvbq\") pod \"kube-auth-proxy-5688cff95c-j5665\" (UID: \"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f\") " pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.194916 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.194884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:19.215719 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.215673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665"
Jul 24 15:47:19.294760 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.294725    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:19.358608 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.358567    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5688cff95c-j5665"]
Jul 24 15:47:19.361752 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:47:19.361722    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6d8bde_deb3_4bb1_ae5a_cfa24ec2d36f.slice/crio-d092dbb1f3df280eb691deca778454265fa6716ec74b8415059c3432c4168d64 WatchSource:0}: Error finding container d092dbb1f3df280eb691deca778454265fa6716ec74b8415059c3432c4168d64: Status 404 returned error can't find the container with id d092dbb1f3df280eb691deca778454265fa6716ec74b8415059c3432c4168d64
Jul 24 15:47:19.534952 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.534918    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665" event={"ID":"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f","Type":"ContainerStarted","Data":"d092dbb1f3df280eb691deca778454265fa6716ec74b8415059c3432c4168d64"}
Jul 24 15:47:19.802281 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.802241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:19.958747 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:19.958698    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:20.085392 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:20.085303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:20.200573 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:20.200531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:21.102398 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:21.102362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:22.548120 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:22.548075    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665" event={"ID":"1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f","Type":"ContainerStarted","Data":"ac8257babcdd747f97f2ef1bd0c048d572da181c458587cb205c3036a7d0ab5c"}
Jul 24 15:47:22.564464 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:22.564411    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5688cff95c-j5665" podStartSLOduration=1.9664934710000002 podStartE2EDuration="4.564393841s" totalImagesPullingTime="2.59790037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:22.563390413 +0000 UTC m=+384.115630186" watchObservedRunningTime="2026-07-24 15:47:22.564393841 +0000 UTC m=+384.116633619"
Jul 24 15:47:25.603987 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:25.603949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:25.762362 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:25.762325    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:26.074332 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.074294    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"]
Jul 24 15:47:26.077395 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.077378    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.079899 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.079872    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 15:47:26.080032 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.079901    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9pcqq\""
Jul 24 15:47:26.080032 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.079911    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 15:47:26.080032 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.079913    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 15:47:26.080354 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.080338    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 15:47:26.095778 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.095753    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"]
Jul 24 15:47:26.206937 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.206899    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.206937 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.206940    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.207201 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.206976    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2m4m\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-kube-api-access-w2m4m\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.207201 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.207006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.207201 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.207046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.207201 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.207153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.207354 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.207202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307630 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2m4m\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-kube-api-access-w2m4m\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.307829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.308087 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.307830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.308805 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.308722    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.310826 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.310800    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.310930 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.310894    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.310996 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.310972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.311460 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.311441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.322909 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.322871    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.326527 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.326461    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2m4m\" (UniqueName: \"kubernetes.io/projected/b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6-kube-api-access-w2m4m\") pod \"istiod-openshift-gateway-5677ccb49b-rr4wf\" (UID: \"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.387215 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.387184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:26.427710 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.427671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:26.531423 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.531370    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"]
Jul 24 15:47:26.534233 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:47:26.534203    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d4fb3f_caf4_4a3d_b6a5_dd5d5269eec6.slice/crio-86750b8255a0c387cb76205151edb6894017b6050254c87e6e3615293d8be15c WatchSource:0}: Error finding container 86750b8255a0c387cb76205151edb6894017b6050254c87e6e3615293d8be15c: Status 404 returned error can't find the container with id 86750b8255a0c387cb76205151edb6894017b6050254c87e6e3615293d8be15c
Jul 24 15:47:26.564685 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.564654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf" event={"ID":"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6","Type":"ContainerStarted","Data":"86750b8255a0c387cb76205151edb6894017b6050254c87e6e3615293d8be15c"}
Jul 24 15:47:26.653017 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.652883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:26.945535 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:26.945421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:29.030707 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.030667    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:47:29.031068 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.030737    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:47:29.146829 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.146777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:29.243394 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.243363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:29.352303 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.352218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:29.463605 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.463572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:29.581576 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.581536    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf" event={"ID":"b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6","Type":"ContainerStarted","Data":"df32b3a3bb1c20a6bfbbdd2acc749279a8329770651cd9349d6d933bc0062059"}
Jul 24 15:47:29.581773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.581645    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:29.619184 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:29.619036    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf" podStartSLOduration=1.12489044 podStartE2EDuration="3.619016261s" totalImagesPullingTime="2.494125821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:29.615397082 +0000 UTC m=+391.167636855" watchObservedRunningTime="2026-07-24 15:47:29.619016261 +0000 UTC m=+391.171256036"
Jul 24 15:47:30.457688 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:30.457658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:30.554983 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:30.554935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:30.587732 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:30.587703    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rr4wf"
Jul 24 15:47:30.665652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:30.665616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:47:43.813616 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:47:43.813574    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.470631 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:19.470592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.572141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:19.572092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:19.670116 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:19.670036    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:20.166914 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.166881    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fvbsf"]
Jul 24 15:48:20.176745 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.176715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:20.179382 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.179345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fvbsf"]
Jul 24 15:48:20.179780 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.179753    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pn9ll\""
Jul 24 15:48:20.179913 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.179765    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 15:48:20.180731 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.180708    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 15:48:20.330949 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.330907    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2thn\" (UniqueName: \"kubernetes.io/projected/e59191f3-f7c6-4054-b868-de0fc590df77-kube-api-access-n2thn\") pod \"kuadrant-operator-catalog-fvbsf\" (UID: \"e59191f3-f7c6-4054-b868-de0fc590df77\") " pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:20.411878 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.411844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:20.432082 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.431974    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2thn\" (UniqueName: \"kubernetes.io/projected/e59191f3-f7c6-4054-b868-de0fc590df77-kube-api-access-n2thn\") pod \"kuadrant-operator-catalog-fvbsf\" (UID: \"e59191f3-f7c6-4054-b868-de0fc590df77\") " pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:20.442016 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.441984    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2thn\" (UniqueName: \"kubernetes.io/projected/e59191f3-f7c6-4054-b868-de0fc590df77-kube-api-access-n2thn\") pod \"kuadrant-operator-catalog-fvbsf\" (UID: \"e59191f3-f7c6-4054-b868-de0fc590df77\") " pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:20.488119 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.488084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:20.607000 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.606835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:20.633329 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.633305    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fvbsf"]
Jul 24 15:48:20.635808 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:48:20.635785    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59191f3_f7c6_4054_b868_de0fc590df77.slice/crio-02b88a41d3ff04c7e83661b7d063a6e64c7fb0453be775ce3e31f635b95584ea WatchSource:0}: Error finding container 02b88a41d3ff04c7e83661b7d063a6e64c7fb0453be775ce3e31f635b95584ea: Status 404 returned error can't find the container with id 02b88a41d3ff04c7e83661b7d063a6e64c7fb0453be775ce3e31f635b95584ea
Jul 24 15:48:20.767435 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:20.767395    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf" event={"ID":"e59191f3-f7c6-4054-b868-de0fc590df77","Type":"ContainerStarted","Data":"02b88a41d3ff04c7e83661b7d063a6e64c7fb0453be775ce3e31f635b95584ea"}
Jul 24 15:48:22.780021 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:22.779988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf" event={"ID":"e59191f3-f7c6-4054-b868-de0fc590df77","Type":"ContainerStarted","Data":"605919b40edae689143e38abcbb5ef1f7d2c65e210ced34e70c2bb4e425093b9"}
Jul 24 15:48:22.810591 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:22.810530    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf" podStartSLOduration=0.783078524 podStartE2EDuration="2.810509298s" totalImagesPullingTime="2.027430774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:22.8083815 +0000 UTC m=+444.360621272" watchObservedRunningTime="2026-07-24 15:48:22.810509298 +0000 UTC m=+444.362749070"
Jul 24 15:48:30.489254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:30.489205    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:30.489857 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:30.489280    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:30.511901 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:30.511869    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:30.833997 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:30.833918    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-fvbsf"
Jul 24 15:48:43.115802 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:43.115761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:43.517560 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:43.517527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:44.663849 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:44.663811    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:46.486372 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.486341    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"]
Jul 24 15:48:46.489897 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.489874    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:46.492850 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.492825    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zs6kp\""
Jul 24 15:48:46.502824 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.502797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"]
Jul 24 15:48:46.539391 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.539348    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfgbj\" (UniqueName: \"kubernetes.io/projected/5f084a21-e96c-43cd-af71-d0881e61caea-kube-api-access-bfgbj\") pod \"authorino-operator-75cc8d5cb7-kn4xq\" (UID: \"5f084a21-e96c-43cd-af71-d0881e61caea\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:46.640219 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.640183    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfgbj\" (UniqueName: \"kubernetes.io/projected/5f084a21-e96c-43cd-af71-d0881e61caea-kube-api-access-bfgbj\") pod \"authorino-operator-75cc8d5cb7-kn4xq\" (UID: \"5f084a21-e96c-43cd-af71-d0881e61caea\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:46.659122 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.659080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfgbj\" (UniqueName: \"kubernetes.io/projected/5f084a21-e96c-43cd-af71-d0881e61caea-kube-api-access-bfgbj\") pod \"authorino-operator-75cc8d5cb7-kn4xq\" (UID: \"5f084a21-e96c-43cd-af71-d0881e61caea\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:46.799781 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.799687    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:46.945927 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:48:46.945889    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f084a21_e96c_43cd_af71_d0881e61caea.slice/crio-b300c2ced882237b00a3124bfabc52bd3d080e7a8098a9390fc84c70f3f12490 WatchSource:0}: Error finding container b300c2ced882237b00a3124bfabc52bd3d080e7a8098a9390fc84c70f3f12490: Status 404 returned error can't find the container with id b300c2ced882237b00a3124bfabc52bd3d080e7a8098a9390fc84c70f3f12490
Jul 24 15:48:46.949896 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:46.949874    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"]
Jul 24 15:48:47.881369 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:47.881318    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq" event={"ID":"5f084a21-e96c-43cd-af71-d0881e61caea","Type":"ContainerStarted","Data":"b300c2ced882237b00a3124bfabc52bd3d080e7a8098a9390fc84c70f3f12490"}
Jul 24 15:48:48.140420 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:48.140276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:49.891721 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:49.891673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq" event={"ID":"5f084a21-e96c-43cd-af71-d0881e61caea","Type":"ContainerStarted","Data":"db1209f6faf5ccb6567e5b9f2d866769236a1cb4c87cc442561450a3560102c4"}
Jul 24 15:48:49.892221 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:49.891768    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:48:49.910151 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:49.910078    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq" podStartSLOduration=1.450122286 podStartE2EDuration="3.910061694s" totalImagesPullingTime="2.459939408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:49.908916549 +0000 UTC m=+471.461156328" watchObservedRunningTime="2026-07-24 15:48:49.910061694 +0000 UTC m=+471.462301467"
Jul 24 15:48:51.115703 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:51.115665    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:51.801979 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:51.801945    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:52.221354 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:52.221266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:52.325704 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:52.325662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:53.209245 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:53.209209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:55.571995 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:55.571955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.189303 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.189265    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:57.594910 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.594875    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"]
Jul 24 15:48:57.603212 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.603185    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:57.606425 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.606405    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m4gz4\""
Jul 24 15:48:57.611862 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.611834    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"]
Jul 24 15:48:57.616689 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.616656    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf5ps\" (UniqueName: \"kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" (UID: \"1f2d992e-671a-4353-96b9-c711d1a1aa78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:57.717935 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.717889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf5ps\" (UniqueName: \"kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" (UID: \"1f2d992e-671a-4353-96b9-c711d1a1aa78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:57.732792 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.732760    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf5ps\" (UniqueName: \"kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" (UID: \"1f2d992e-671a-4353-96b9-c711d1a1aa78\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:57.913950 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:57.913859    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:58.065052 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:58.065025    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"]
Jul 24 15:48:58.067707 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:48:58.067679    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2d992e_671a_4353_96b9_c711d1a1aa78.slice/crio-c5334a0b5d3302ffb6c0052f66fa00b2bb021749ffe69ace4405d8494a5b9046 WatchSource:0}: Error finding container c5334a0b5d3302ffb6c0052f66fa00b2bb021749ffe69ace4405d8494a5b9046: Status 404 returned error can't find the container with id c5334a0b5d3302ffb6c0052f66fa00b2bb021749ffe69ace4405d8494a5b9046
Jul 24 15:48:58.842949 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:58.842901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:58.930613 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:58.930571    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx" event={"ID":"1f2d992e-671a-4353-96b9-c711d1a1aa78","Type":"ContainerStarted","Data":"c5334a0b5d3302ffb6c0052f66fa00b2bb021749ffe69ace4405d8494a5b9046"}
Jul 24 15:48:58.970118 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:58.970080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:59.234200 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:59.234160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:59.388249 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:59.388209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:48:59.936221 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:59.936177    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx" event={"ID":"1f2d992e-671a-4353-96b9-c711d1a1aa78","Type":"ContainerStarted","Data":"9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375"}
Jul 24 15:48:59.936589 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:59.936307    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:48:59.982870 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:48:59.982814    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx" podStartSLOduration=1.256206042 podStartE2EDuration="2.98279826s" totalImagesPullingTime="1.726592218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:59.982078946 +0000 UTC m=+481.534318720" watchObservedRunningTime="2026-07-24 15:48:59.98279826 +0000 UTC m=+481.535038033"
Jul 24 15:49:00.897825 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:00.897796    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kn4xq"
Jul 24 15:49:02.999196 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:02.999143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.105753 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:03.105705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.530656 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:03.530618    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.656481 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:03.656446    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.746995 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:03.746964    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:03.944438 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:03.944354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:08.341483 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:08.341449    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:08.443800 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:08.443765    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:10.948366 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:10.948328    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:11.263288 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:11.263251    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:11.384781 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:11.384738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:14.403915 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.403883    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"]
Jul 24 15:49:14.404452 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.404065    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" containerName="manager" containerID="cri-o://9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375" gracePeriod=2
Jul 24 15:49:14.434321 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.433896    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"]
Jul 24 15:49:14.449477 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:49:14.449422    2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"1f2d992e-671a-4353-96b9-c711d1a1aa78\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-vq7dx\": pods \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:14.495324 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.495286    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"]
Jul 24 15:49:14.495620 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.495606    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" containerName="manager"
Jul 24 15:49:14.495662 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.495622    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" containerName="manager"
Jul 24 15:49:14.495694 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.495662    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" containerName="manager"
Jul 24 15:49:14.498464 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.498448    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:14.500739 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:49:14.500700    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" is forbidden: User \"system:node:ip-10-0-133-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-134.ec2.internal' and this object" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:14.524370 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.524336    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"]
Jul 24 15:49:14.559263 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.559229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j2xt\" (UniqueName: \"kubernetes.io/projected/cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e-kube-api-access-6j2xt\") pod \"limitador-operator-controller-manager-859d7b78bd-45972\" (UID: \"cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:14.644410 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.644386    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:14.650688 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:49:14.650653    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-vq7dx\" is forbidden: User \"system:node:ip-10-0-133-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-134.ec2.internal' and this object" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:14.660041 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.659949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j2xt\" (UniqueName: \"kubernetes.io/projected/cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e-kube-api-access-6j2xt\") pod \"limitador-operator-controller-manager-859d7b78bd-45972\" (UID: \"cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:14.679054 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.679022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j2xt\" (UniqueName: \"kubernetes.io/projected/cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e-kube-api-access-6j2xt\") pod \"limitador-operator-controller-manager-859d7b78bd-45972\" (UID: \"cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:14.760858 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.760818    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf5ps\" (UniqueName: \"kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps\") pod \"1f2d992e-671a-4353-96b9-c711d1a1aa78\" (UID: \"1f2d992e-671a-4353-96b9-c711d1a1aa78\") "
Jul 24 15:49:14.763218 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.763184    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps" (OuterVolumeSpecName: "kube-api-access-mf5ps") pod "1f2d992e-671a-4353-96b9-c711d1a1aa78" (UID: "1f2d992e-671a-4353-96b9-c711d1a1aa78"). InnerVolumeSpecName "kube-api-access-mf5ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:14.809045 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.809001    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:14.862267 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.862218    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf5ps\" (UniqueName: \"kubernetes.io/projected/1f2d992e-671a-4353-96b9-c711d1a1aa78-kube-api-access-mf5ps\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:49:14.949240 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.949160    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2d992e-671a-4353-96b9-c711d1a1aa78" path="/var/lib/kubelet/pods/1f2d992e-671a-4353-96b9-c711d1a1aa78/volumes"
Jul 24 15:49:14.951344 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:14.951313    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"]
Jul 24 15:49:14.955037 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:49:14.955002    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd67c53_6da0_4ff2_a9c1_fd04d32efe7e.slice/crio-a9c91941129622e6fd8d91e81466fcd6edd0b307889ae67671a8ab0ac735f6ef WatchSource:0}: Error finding container a9c91941129622e6fd8d91e81466fcd6edd0b307889ae67671a8ab0ac735f6ef: Status 404 returned error can't find the container with id a9c91941129622e6fd8d91e81466fcd6edd0b307889ae67671a8ab0ac735f6ef
Jul 24 15:49:15.001115 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.001083    2567 generic.go:356] "Generic (PLEG): container finished" podID="1f2d992e-671a-4353-96b9-c711d1a1aa78" containerID="9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375" exitCode=0
Jul 24 15:49:15.001115 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.001145    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7dx"
Jul 24 15:49:15.001343 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.001223    2567 scope.go:117] "RemoveContainer" containerID="9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375"
Jul 24 15:49:15.002618 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.002591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972" event={"ID":"cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e","Type":"ContainerStarted","Data":"a9c91941129622e6fd8d91e81466fcd6edd0b307889ae67671a8ab0ac735f6ef"}
Jul 24 15:49:15.015063 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.015043    2567 scope.go:117] "RemoveContainer" containerID="9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375"
Jul 24 15:49:15.015421 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:49:15.015390    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375\": container with ID starting with 9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375 not found: ID does not exist" containerID="9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375"
Jul 24 15:49:15.015499 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:15.015429    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375"} err="failed to get container status \"9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375\": rpc error: code = NotFound desc = could not find container \"9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375\": container with ID starting with 9f98df17822da985b2c3e709400be8372e32b84c7edb3f1252c60acd8a043375 not found: ID does not exist"
Jul 24 15:49:16.009471 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:16.009437    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972" event={"ID":"cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e","Type":"ContainerStarted","Data":"e92b81f489ec316c6b690d355decdf4ae6d72f52465b6a43559df5b3dc7e9be9"}
Jul 24 15:49:16.009856 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:16.009562    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:16.040646 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:16.040504    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972" podStartSLOduration=2.040485688 podStartE2EDuration="2.040485688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:16.039233484 +0000 UTC m=+497.591473257" watchObservedRunningTime="2026-07-24 15:49:16.040485688 +0000 UTC m=+497.592725462"
Jul 24 15:49:17.441083 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:17.441053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:17.615557 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:17.615521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:21.340784 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:21.340748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:21.442141 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:21.442084    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:22.022441 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:22.022406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:22.205466 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:22.205427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:24.630326 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:24.630295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:24.763984 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:24.763942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:27.015751 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:27.015722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-45972"
Jul 24 15:49:29.925598 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:29.925551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:30.025652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:30.025620    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:37.752628 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:37.752572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:37.862665 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:37.862627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:43.868322 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:43.868283    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:43.976989 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:43.976953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:45.512062 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:45.512027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:45.623484 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:45.623443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:45.689329 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:45.689295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:46.386111 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.386065    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:46.454497 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.454445    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:49:46.455946 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.455913    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:49:46.464763 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.464729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:49:46.464926 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.464856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:49:46.467376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.467352    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cw7rl\""
Jul 24 15:49:46.629038 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.629001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmcv\" (UniqueName: \"kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv\") pod \"authorino-75d556bd77-kzfkq\" (UID: \"8607a251-ed04-43f7-b218-3a6804a4c00c\") " pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:49:46.729568 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.729476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmcv\" (UniqueName: \"kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv\") pod \"authorino-75d556bd77-kzfkq\" (UID: \"8607a251-ed04-43f7-b218-3a6804a4c00c\") " pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:49:46.738503 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.738468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmcv\" (UniqueName: \"kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv\") pod \"authorino-75d556bd77-kzfkq\" (UID: \"8607a251-ed04-43f7-b218-3a6804a4c00c\") " pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:49:46.774546 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.774513    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:49:46.908406 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:46.908368    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:49:46.911678 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:49:46.911653    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8607a251_ed04_43f7_b218_3a6804a4c00c.slice/crio-3a0d6bd132528d38cc4391d5f3f04127c33a15ddf3fc3db1a0d56d74db31c68f WatchSource:0}: Error finding container 3a0d6bd132528d38cc4391d5f3f04127c33a15ddf3fc3db1a0d56d74db31c68f: Status 404 returned error can't find the container with id 3a0d6bd132528d38cc4391d5f3f04127c33a15ddf3fc3db1a0d56d74db31c68f
Jul 24 15:49:47.141361 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:47.141323    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kzfkq" event={"ID":"8607a251-ed04-43f7-b218-3a6804a4c00c","Type":"ContainerStarted","Data":"3a0d6bd132528d38cc4391d5f3f04127c33a15ddf3fc3db1a0d56d74db31c68f"}
Jul 24 15:49:50.157472 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:50.157439    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kzfkq" event={"ID":"8607a251-ed04-43f7-b218-3a6804a4c00c","Type":"ContainerStarted","Data":"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"}
Jul 24 15:49:50.175752 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:49:50.175694    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kzfkq" podStartSLOduration=1.017803828 podStartE2EDuration="4.175670085s" totalImagesPullingTime="3.157866257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:50.174813421 +0000 UTC m=+531.727053195" watchObservedRunningTime="2026-07-24 15:49:50.175670085 +0000 UTC m=+531.727909858"
Jul 24 15:50:06.841390 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:06.841352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:06.935419 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:06.935372    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:07.345079 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:07.345033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:19.992887 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:19.992852    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:20.118825 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.118794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:20.493220 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.493191    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-594cdb5fb5-9zn8j"]
Jul 24 15:50:20.537913 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.537876    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-594cdb5fb5-9zn8j"]
Jul 24 15:50:20.538116 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.538006    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:20.584888 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.584848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8zvl\" (UniqueName: \"kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl\") pod \"authorino-594cdb5fb5-9zn8j\" (UID: \"55f4cc56-81e9-4e2b-85db-143e35479e95\") " pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:20.665030 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.664991    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594cdb5fb5-9zn8j"]
Jul 24 15:50:20.665293 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:50:20.665273    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x8zvl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-594cdb5fb5-9zn8j" podUID="55f4cc56-81e9-4e2b-85db-143e35479e95"
Jul 24 15:50:20.685901 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.685869    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8zvl\" (UniqueName: \"kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl\") pod \"authorino-594cdb5fb5-9zn8j\" (UID: \"55f4cc56-81e9-4e2b-85db-143e35479e95\") " pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:20.696148 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:20.696105    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8zvl\" (UniqueName: \"kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl\") pod \"authorino-594cdb5fb5-9zn8j\" (UID: \"55f4cc56-81e9-4e2b-85db-143e35479e95\") " pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:21.281615 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:21.281579    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:21.287721 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:21.287694    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:21.393083 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:21.393034    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8zvl\" (UniqueName: \"kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl\") pod \"55f4cc56-81e9-4e2b-85db-143e35479e95\" (UID: \"55f4cc56-81e9-4e2b-85db-143e35479e95\") "
Jul 24 15:50:21.395398 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:21.395366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl" (OuterVolumeSpecName: "kube-api-access-x8zvl") pod "55f4cc56-81e9-4e2b-85db-143e35479e95" (UID: "55f4cc56-81e9-4e2b-85db-143e35479e95"). InnerVolumeSpecName "kube-api-access-x8zvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:50:21.493626 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:21.493588    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8zvl\" (UniqueName: \"kubernetes.io/projected/55f4cc56-81e9-4e2b-85db-143e35479e95-kube-api-access-x8zvl\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:50:22.286118 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.286081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-594cdb5fb5-9zn8j"
Jul 24 15:50:22.315429 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.315394    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-594cdb5fb5-9zn8j"]
Jul 24 15:50:22.319841 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.319802    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-594cdb5fb5-9zn8j"]
Jul 24 15:50:22.794505 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.794467    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:50:22.794684 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.794645    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kzfkq" podUID="8607a251-ed04-43f7-b218-3a6804a4c00c" containerName="authorino" containerID="cri-o://2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710" gracePeriod=30
Jul 24 15:50:22.950140 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:22.950087    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f4cc56-81e9-4e2b-85db-143e35479e95" path="/var/lib/kubelet/pods/55f4cc56-81e9-4e2b-85db-143e35479e95/volumes"
Jul 24 15:50:23.047547 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.047474    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:50:23.108911 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.108872    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsmcv\" (UniqueName: \"kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv\") pod \"8607a251-ed04-43f7-b218-3a6804a4c00c\" (UID: \"8607a251-ed04-43f7-b218-3a6804a4c00c\") "
Jul 24 15:50:23.111189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.111150    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv" (OuterVolumeSpecName: "kube-api-access-qsmcv") pod "8607a251-ed04-43f7-b218-3a6804a4c00c" (UID: "8607a251-ed04-43f7-b218-3a6804a4c00c"). InnerVolumeSpecName "kube-api-access-qsmcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:50:23.209695 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.209635    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsmcv\" (UniqueName: \"kubernetes.io/projected/8607a251-ed04-43f7-b218-3a6804a4c00c-kube-api-access-qsmcv\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:50:23.291995 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.291964    2567 generic.go:356] "Generic (PLEG): container finished" podID="8607a251-ed04-43f7-b218-3a6804a4c00c" containerID="2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710" exitCode=0
Jul 24 15:50:23.292431 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.292006    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kzfkq"
Jul 24 15:50:23.292431 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.292006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kzfkq" event={"ID":"8607a251-ed04-43f7-b218-3a6804a4c00c","Type":"ContainerDied","Data":"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"}
Jul 24 15:50:23.292431 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.292148    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kzfkq" event={"ID":"8607a251-ed04-43f7-b218-3a6804a4c00c","Type":"ContainerDied","Data":"3a0d6bd132528d38cc4391d5f3f04127c33a15ddf3fc3db1a0d56d74db31c68f"}
Jul 24 15:50:23.292431 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.292172    2567 scope.go:117] "RemoveContainer" containerID="2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"
Jul 24 15:50:23.300639 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.300620    2567 scope.go:117] "RemoveContainer" containerID="2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"
Jul 24 15:50:23.300958 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:50:23.300939    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710\": container with ID starting with 2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710 not found: ID does not exist" containerID="2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"
Jul 24 15:50:23.301042 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.300967    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710"} err="failed to get container status \"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710\": rpc error: code = NotFound desc = could not find container \"2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710\": container with ID starting with 2d4ce34a4552000009d51a295dc549f90b9dcd7a0d155539e47388e6a3750710 not found: ID does not exist"
Jul 24 15:50:23.311491 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.311453    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:50:23.317244 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:23.317217    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kzfkq"]
Jul 24 15:50:24.953847 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:24.953810    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8607a251-ed04-43f7-b218-3a6804a4c00c" path="/var/lib/kubelet/pods/8607a251-ed04-43f7-b218-3a6804a4c00c/volumes"
Jul 24 15:50:27.608478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.608435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:27.625555 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.625522    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:27.625816 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.625805    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8607a251-ed04-43f7-b218-3a6804a4c00c" containerName="authorino"
Jul 24 15:50:27.625864 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.625818    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8607a251-ed04-43f7-b218-3a6804a4c00c" containerName="authorino"
Jul 24 15:50:27.625898 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.625865    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8607a251-ed04-43f7-b218-3a6804a4c00c" containerName="authorino"
Jul 24 15:50:27.629965 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.629940    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.632504 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.632476    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:50:27.632504 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.632496    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:50:27.632684 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.632520    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdgjn\""
Jul 24 15:50:27.641740 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.641713    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:27.709014 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.708969    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:27.748581 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.748551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.748751 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.748611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdr2r\" (UniqueName: \"kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.808841 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.808807    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:27.815478 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.815450    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:50:27.818738 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.818717    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.829193 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.829165    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:50:27.849766 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.849730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t44r7\" (UniqueName: \"kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.849925 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.849778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.849925 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.849839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdr2r\" (UniqueName: \"kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.849925 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.849856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.852392 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.852361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.858350 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.858328    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdr2r\" (UniqueName: \"kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r\") pod \"maas-controller-5f5f66dd5d-tj5r5\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") " pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.941116 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.941030    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:27.950406 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.950378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.950527 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.950421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t44r7\" (UniqueName: \"kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.952946 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.952922    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:27.959618 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:27.959592    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t44r7\" (UniqueName: \"kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7\") pod \"maas-controller-64456c74cb-rf87q\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") " pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:28.073921 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:28.073890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:28.076083 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:50:28.076057    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdc8210_d197_46df_aca9_de63c6f598ea.slice/crio-4723ff46fc534c8a1bb4642e307403ab26da618b5dad223eb105de77941b2a42 WatchSource:0}: Error finding container 4723ff46fc534c8a1bb4642e307403ab26da618b5dad223eb105de77941b2a42: Status 404 returned error can't find the container with id 4723ff46fc534c8a1bb4642e307403ab26da618b5dad223eb105de77941b2a42
Jul 24 15:50:28.129512 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:28.129474    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:28.265815 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:28.265786    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:50:28.269696 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:50:28.269672    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbfe1e00_27ff_45fe_b0ef_8404ef69a7a9.slice/crio-b9dba89520fbb9c3e9e85c9a677a1cb1b173c3b087564f43afc879242d09b8e1 WatchSource:0}: Error finding container b9dba89520fbb9c3e9e85c9a677a1cb1b173c3b087564f43afc879242d09b8e1: Status 404 returned error can't find the container with id b9dba89520fbb9c3e9e85c9a677a1cb1b173c3b087564f43afc879242d09b8e1
Jul 24 15:50:28.316886 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:28.316848    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" event={"ID":"abdc8210-d197-46df-aca9-de63c6f598ea","Type":"ContainerStarted","Data":"4723ff46fc534c8a1bb4642e307403ab26da618b5dad223eb105de77941b2a42"}
Jul 24 15:50:28.317974 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:28.317950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-rf87q" event={"ID":"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9","Type":"ContainerStarted","Data":"b9dba89520fbb9c3e9e85c9a677a1cb1b173c3b087564f43afc879242d09b8e1"}
Jul 24 15:50:30.942003 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:30.941966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:31.071703 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.071615    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:50:31.337465 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.337375    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" event={"ID":"abdc8210-d197-46df-aca9-de63c6f598ea","Type":"ContainerStarted","Data":"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"}
Jul 24 15:50:31.337618 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.337494    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:31.338861 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.338839    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-rf87q" event={"ID":"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9","Type":"ContainerStarted","Data":"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"}
Jul 24 15:50:31.338990 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.338977    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:31.360963 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.360912    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" podStartSLOduration=1.666176036 podStartE2EDuration="4.360896758s" totalImagesPullingTime="2.694720722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:50:31.359512792 +0000 UTC m=+572.911752565" watchObservedRunningTime="2026-07-24 15:50:31.360896758 +0000 UTC m=+572.913136531"
Jul 24 15:50:31.390356 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:31.390300    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64456c74cb-rf87q" podStartSLOduration=1.88257531 podStartE2EDuration="4.390285191s" totalImagesPullingTime="2.507709881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:50:31.388485296 +0000 UTC m=+572.940725081" watchObservedRunningTime="2026-07-24 15:50:31.390285191 +0000 UTC m=+572.942524963"
Jul 24 15:50:42.346319 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.346287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:50:42.346814 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.346374    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:42.406479 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.406444    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:42.406685 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.406659    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" podUID="abdc8210-d197-46df-aca9-de63c6f598ea" containerName="manager" containerID="cri-o://f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50" gracePeriod=30
Jul 24 15:50:42.650590 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.650564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:42.780190 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.780154    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert\") pod \"abdc8210-d197-46df-aca9-de63c6f598ea\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") "
Jul 24 15:50:42.780367 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.780206    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdr2r\" (UniqueName: \"kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r\") pod \"abdc8210-d197-46df-aca9-de63c6f598ea\" (UID: \"abdc8210-d197-46df-aca9-de63c6f598ea\") "
Jul 24 15:50:42.782636 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.782606    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r" (OuterVolumeSpecName: "kube-api-access-mdr2r") pod "abdc8210-d197-46df-aca9-de63c6f598ea" (UID: "abdc8210-d197-46df-aca9-de63c6f598ea"). InnerVolumeSpecName "kube-api-access-mdr2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:50:42.782636 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.782622    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "abdc8210-d197-46df-aca9-de63c6f598ea" (UID: "abdc8210-d197-46df-aca9-de63c6f598ea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:50:42.881406 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.881371    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/abdc8210-d197-46df-aca9-de63c6f598ea-webhook-cert\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:50:42.881406 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:42.881400    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdr2r\" (UniqueName: \"kubernetes.io/projected/abdc8210-d197-46df-aca9-de63c6f598ea-kube-api-access-mdr2r\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:50:43.389701 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.389663    2567 generic.go:356] "Generic (PLEG): container finished" podID="abdc8210-d197-46df-aca9-de63c6f598ea" containerID="f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50" exitCode=0
Jul 24 15:50:43.390142 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.389713    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5"
Jul 24 15:50:43.390142 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.389747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" event={"ID":"abdc8210-d197-46df-aca9-de63c6f598ea","Type":"ContainerDied","Data":"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"}
Jul 24 15:50:43.390142 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.389804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5f66dd5d-tj5r5" event={"ID":"abdc8210-d197-46df-aca9-de63c6f598ea","Type":"ContainerDied","Data":"4723ff46fc534c8a1bb4642e307403ab26da618b5dad223eb105de77941b2a42"}
Jul 24 15:50:43.390142 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.389827    2567 scope.go:117] "RemoveContainer" containerID="f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"
Jul 24 15:50:43.398107 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.398087    2567 scope.go:117] "RemoveContainer" containerID="f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"
Jul 24 15:50:43.398455 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:50:43.398438    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50\": container with ID starting with f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50 not found: ID does not exist" containerID="f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"
Jul 24 15:50:43.398492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.398465    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50"} err="failed to get container status \"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50\": rpc error: code = NotFound desc = could not find container \"f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50\": container with ID starting with f5363073ddabfb2abc12bf12ef45ed02adaa27e332b4e5af88b191edf388ad50 not found: ID does not exist"
Jul 24 15:50:43.406198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.406169    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:43.409591 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:43.409560    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f5f66dd5d-tj5r5"]
Jul 24 15:50:44.952891 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:50:44.952849    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abdc8210-d197-46df-aca9-de63c6f598ea" path="/var/lib/kubelet/pods/abdc8210-d197-46df-aca9-de63c6f598ea/volumes"
Jul 24 15:51:02.687255 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:02.687206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:02.785060 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:02.784985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:03.434984 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.434943    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:03.563446 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.563406    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"]
Jul 24 15:51:03.563835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.563820    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abdc8210-d197-46df-aca9-de63c6f598ea" containerName="manager"
Jul 24 15:51:03.563883 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.563838    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="abdc8210-d197-46df-aca9-de63c6f598ea" containerName="manager"
Jul 24 15:51:03.563927 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.563916    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="abdc8210-d197-46df-aca9-de63c6f598ea" containerName="manager"
Jul 24 15:51:03.572731 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.572697    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.577184 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.577154    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 15:51:03.577339 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.577208    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"]
Jul 24 15:51:03.577339 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.577324    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 24 15:51:03.577445 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.577395    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zhdk6\""
Jul 24 15:51:03.577532 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.577171    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:51:03.720181 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.720068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:03.754636 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.754601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5af7435b-34c8-4d7a-a8b6-955fc0f65700-maas-api-tls\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.754636 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.754635    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af7435b-34c8-4d7a-a8b6-955fc0f65700-openshift-service-ca\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.754851 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.754673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxh5l\" (UniqueName: \"kubernetes.io/projected/5af7435b-34c8-4d7a-a8b6-955fc0f65700-kube-api-access-rxh5l\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.855638 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.855600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5af7435b-34c8-4d7a-a8b6-955fc0f65700-maas-api-tls\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.855638 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.855638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af7435b-34c8-4d7a-a8b6-955fc0f65700-openshift-service-ca\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.855904 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.855671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxh5l\" (UniqueName: \"kubernetes.io/projected/5af7435b-34c8-4d7a-a8b6-955fc0f65700-kube-api-access-rxh5l\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.856387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.856362    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af7435b-34c8-4d7a-a8b6-955fc0f65700-openshift-service-ca\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.858296 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.858272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5af7435b-34c8-4d7a-a8b6-955fc0f65700-maas-api-tls\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.863652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.863629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxh5l\" (UniqueName: \"kubernetes.io/projected/5af7435b-34c8-4d7a-a8b6-955fc0f65700-kube-api-access-rxh5l\") pod \"maas-api-78bb94d85f-6pkr6\" (UID: \"5af7435b-34c8-4d7a-a8b6-955fc0f65700\") " pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.884289 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.884256    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:03.945218 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:03.945180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:04.032093 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.032061    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"]
Jul 24 15:51:04.034803 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.034758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:04.035846 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:51:04.035822    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af7435b_34c8_4d7a_a8b6_955fc0f65700.slice/crio-4867c1995d17122cd0da839823ebeb28d2802536d060168d76c41b28c676eaf6 WatchSource:0}: Error finding container 4867c1995d17122cd0da839823ebeb28d2802536d060168d76c41b28c676eaf6: Status 404 returned error can't find the container with id 4867c1995d17122cd0da839823ebeb28d2802536d060168d76c41b28c676eaf6
Jul 24 15:51:04.059714 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.059684    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"]
Jul 24 15:51:04.063902 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.063887    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.066689 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.066666    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 15:51:04.066813 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.066719    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ww5p6\""
Jul 24 15:51:04.073962 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.073934    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"]
Jul 24 15:51:04.259751 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.259712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c89a27-1524-4016-9c7c-f03f7634db7e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.259751 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.259758    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcsgd\" (UniqueName: \"kubernetes.io/projected/96c89a27-1524-4016-9c7c-f03f7634db7e-kube-api-access-kcsgd\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.360352 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.360317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c89a27-1524-4016-9c7c-f03f7634db7e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.360552 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.360365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcsgd\" (UniqueName: \"kubernetes.io/projected/96c89a27-1524-4016-9c7c-f03f7634db7e-kube-api-access-kcsgd\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.361032 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.361003    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96c89a27-1524-4016-9c7c-f03f7634db7e-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.368315 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.368292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcsgd\" (UniqueName: \"kubernetes.io/projected/96c89a27-1524-4016-9c7c-f03f7634db7e-kube-api-access-kcsgd\") pod \"payload-pre-processing-687dbb6cf6-bpc8w\" (UID: \"96c89a27-1524-4016-9c7c-f03f7634db7e\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.375090 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.375060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:04.480816 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.480772    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6" event={"ID":"5af7435b-34c8-4d7a-a8b6-955fc0f65700","Type":"ContainerStarted","Data":"4867c1995d17122cd0da839823ebeb28d2802536d060168d76c41b28c676eaf6"}
Jul 24 15:51:04.517162 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:04.517114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"]
Jul 24 15:51:04.519688 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:51:04.519654    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c89a27_1524_4016_9c7c_f03f7634db7e.slice/crio-e386ec79d90901d745789c8299dd19dc613c474f7a6521b24691bdc34541015e WatchSource:0}: Error finding container e386ec79d90901d745789c8299dd19dc613c474f7a6521b24691bdc34541015e: Status 404 returned error can't find the container with id e386ec79d90901d745789c8299dd19dc613c474f7a6521b24691bdc34541015e
Jul 24 15:51:05.488873 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:05.488832    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w" event={"ID":"96c89a27-1524-4016-9c7c-f03f7634db7e","Type":"ContainerStarted","Data":"e386ec79d90901d745789c8299dd19dc613c474f7a6521b24691bdc34541015e"}
Jul 24 15:51:07.499121 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.499085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w" event={"ID":"96c89a27-1524-4016-9c7c-f03f7634db7e","Type":"ContainerStarted","Data":"0bafb671babe157544bcde088a868ff34ab5345d81f8591d62e5b042dbac6a59"}
Jul 24 15:51:07.499579 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.499201    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:07.500749 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.500724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6" event={"ID":"5af7435b-34c8-4d7a-a8b6-955fc0f65700","Type":"ContainerStarted","Data":"ea26a28c4d9298f3dd50c23bec795d6cf8a499e6126d928add5d122807c75612"}
Jul 24 15:51:07.500881 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.500836    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:07.515022 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.514970    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w" podStartSLOduration=1.241180898 podStartE2EDuration="3.514954369s" totalImagesPullingTime="2.273773471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:07.513620773 +0000 UTC m=+609.065860557" watchObservedRunningTime="2026-07-24 15:51:07.514954369 +0000 UTC m=+609.067194142"
Jul 24 15:51:07.530143 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:07.530082    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6" podStartSLOduration=1.8177831009999998 podStartE2EDuration="4.530066957s" totalImagesPullingTime="2.712283856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:07.528979122 +0000 UTC m=+609.081218895" watchObservedRunningTime="2026-07-24 15:51:07.530066957 +0000 UTC m=+609.082306730"
Jul 24 15:51:13.510159 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:13.510112    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-78bb94d85f-6pkr6"
Jul 24 15:51:14.512441 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:14.512396    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:15.811954 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:15.811917    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:18.507871 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:18.507843    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-bpc8w"
Jul 24 15:51:32.878215 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.878115    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"]
Jul 24 15:51:32.881438 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.881420    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:32.884385 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.884360    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 15:51:32.884385 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.884360    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 15:51:32.885369 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.885346    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 15:51:32.885538 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.885516    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lmqq6\""
Jul 24 15:51:32.891327 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.891301    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"]
Jul 24 15:51:32.906419 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:32.906387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:33.004226 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004185    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.004411 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.004411 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npjcm\" (UniqueName: \"kubernetes.io/projected/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-kube-api-access-npjcm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.004411 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.004515 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004413    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.004515 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.004429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.104893 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.104846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.104893 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.104889    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.104927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.104957    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.104983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npjcm\" (UniqueName: \"kubernetes.io/projected/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-kube-api-access-npjcm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105189 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.105014    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105424 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.105396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.105401    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.105492 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.105460    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.107652 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.107620    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.107787 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.107770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.112956 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.112932    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npjcm\" (UniqueName: \"kubernetes.io/projected/aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5-kube-api-access-npjcm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z\" (UID: \"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.192878 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.192786    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:33.337988 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.337921    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"]
Jul 24 15:51:33.340804 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:51:33.340772    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf50365_50b5_43c7_bd3d_e9cc8e15bbc5.slice/crio-7064e21b37ce7dcca1a8788b93255bcf72fa74e2ee92880f9c4e42318d13c689 WatchSource:0}: Error finding container 7064e21b37ce7dcca1a8788b93255bcf72fa74e2ee92880f9c4e42318d13c689: Status 404 returned error can't find the container with id 7064e21b37ce7dcca1a8788b93255bcf72fa74e2ee92880f9c4e42318d13c689
Jul 24 15:51:33.618068 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:33.618026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z" event={"ID":"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5","Type":"ContainerStarted","Data":"7064e21b37ce7dcca1a8788b93255bcf72fa74e2ee92880f9c4e42318d13c689"}
Jul 24 15:51:34.625199 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:34.625161    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z" event={"ID":"aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5","Type":"ContainerStarted","Data":"a612a1d22d1fd452da521756e727135e0ff7dbf9bf5207ed6ae3cde7612c842b"}
Jul 24 15:51:34.643549 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:34.643490    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z" podStartSLOduration=1.531591744 podStartE2EDuration="2.643473497s" totalImagesPullingTime="1.111881753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:34.641673913 +0000 UTC m=+636.193913686" watchObservedRunningTime="2026-07-24 15:51:34.643473497 +0000 UTC m=+636.195713269"
Jul 24 15:51:43.193671 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:43.193628    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:43.194174 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:43.193696    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:43.206394 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:43.206373    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:43.674980 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:43.674949    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z"
Jul 24 15:51:53.729942 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.729910    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"]
Jul 24 15:51:53.733106 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.733089    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.735880 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.735856    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 24 15:51:53.773380 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.773346    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"]
Jul 24 15:51:53.792961 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.792929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:51:53.887654 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.887835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887666    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.887835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.887835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887698    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.887835 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887803    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e4e438-21d3-43a2-b8db-15f796a04a84-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.887990 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.887844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqh2q\" (UniqueName: \"kubernetes.io/projected/a8e4e438-21d3-43a2-b8db-15f796a04a84-kube-api-access-cqh2q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.988852 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e4e438-21d3-43a2-b8db-15f796a04a84-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989049 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.988961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqh2q\" (UniqueName: \"kubernetes.io/projected/a8e4e438-21d3-43a2-b8db-15f796a04a84-kube-api-access-cqh2q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989335 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.989313    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.989359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.989387 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.989379    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.991402 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.991381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8e4e438-21d3-43a2-b8db-15f796a04a84-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.991630 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.991610    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8e4e438-21d3-43a2-b8db-15f796a04a84-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:53.997513 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:53.997485    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqh2q\" (UniqueName: \"kubernetes.io/projected/a8e4e438-21d3-43a2-b8db-15f796a04a84-kube-api-access-cqh2q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tjgzz\" (UID: \"a8e4e438-21d3-43a2-b8db-15f796a04a84\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:54.044268 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.044233    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:51:54.180937 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.180910    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"]
Jul 24 15:51:54.183563 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:51:54.183534    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e4e438_21d3_43a2_b8db_15f796a04a84.slice/crio-95c617930f01e31af32b852c6886fcac28bdae0eddab3d8fe8b9e86d8f1ce249 WatchSource:0}: Error finding container 95c617930f01e31af32b852c6886fcac28bdae0eddab3d8fe8b9e86d8f1ce249: Status 404 returned error can't find the container with id 95c617930f01e31af32b852c6886fcac28bdae0eddab3d8fe8b9e86d8f1ce249
Jul 24 15:51:54.184981 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.184961    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:51:54.714191 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.714064    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz" event={"ID":"a8e4e438-21d3-43a2-b8db-15f796a04a84","Type":"ContainerStarted","Data":"3b159dded06fc790c7c6d35ed25636022b8514c82a742136e5cfdc4d661777cf"}
Jul 24 15:51:54.714191 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.714153    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz" event={"ID":"a8e4e438-21d3-43a2-b8db-15f796a04a84","Type":"ContainerStarted","Data":"95c617930f01e31af32b852c6886fcac28bdae0eddab3d8fe8b9e86d8f1ce249"}
Jul 24 15:51:54.735703 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:51:54.735654    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz" podStartSLOduration=1.528288047 podStartE2EDuration="1.735636695s" totalImagesPullingTime="207.348648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:54.734183128 +0000 UTC m=+656.286422898" watchObservedRunningTime="2026-07-24 15:51:54.735636695 +0000 UTC m=+656.287876488"
Jul 24 15:52:04.045234 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:04.045193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:52:04.045724 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:04.045260    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:52:04.057718 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:04.057691    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:52:04.771106 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:04.771076    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tjgzz"
Jul 24 15:52:11.178454 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:11.178417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:20.501373 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:20.501311    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:33.205106 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:33.205056    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:42.147024 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:42.146978    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:43.847177 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:43.847145    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"]
Jul 24 15:52:43.851016 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:43.850991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:43.853951 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:43.853918    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 24 15:52:43.859662 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:43.859625    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"]
Jul 24 15:52:43.899838 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:43.899802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:44.029853 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.029816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gplt\" (UniqueName: \"kubernetes.io/projected/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-kube-api-access-6gplt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.030031 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.029869    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.030031 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.029959    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.030031 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.029994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.030031 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.030009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.030213 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.030067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130583 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gplt\" (UniqueName: \"kubernetes.io/projected/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-kube-api-access-6gplt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130583 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130794 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130794 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130794 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.130794 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.130690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.131109 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.131083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.131232 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.131159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.131232 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.131188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.133054 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.133034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.133223 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.133206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.139319 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.139299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gplt\" (UniqueName: \"kubernetes.io/projected/2a8cc48a-0cfc-4458-bf68-97cc7d06af2c-kube-api-access-6gplt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hvtwk\" (UID: \"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.162282 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.162247    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:44.292524 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.292496    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"]
Jul 24 15:52:44.294864 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:52:44.294839    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8cc48a_0cfc_4458_bf68_97cc7d06af2c.slice/crio-2e56e19ff5eebba98c7e7900420446cc0e88abf2bcc97a482ed1b51c5ebc56da WatchSource:0}: Error finding container 2e56e19ff5eebba98c7e7900420446cc0e88abf2bcc97a482ed1b51c5ebc56da: Status 404 returned error can't find the container with id 2e56e19ff5eebba98c7e7900420446cc0e88abf2bcc97a482ed1b51c5ebc56da
Jul 24 15:52:44.932779 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.932744    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk" event={"ID":"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c","Type":"ContainerStarted","Data":"f992739b82e6032f8394d1eb3a66ec9bb9d8368e6998228a1d855221b7021c32"}
Jul 24 15:52:44.933175 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.932800    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk" event={"ID":"2a8cc48a-0cfc-4458-bf68-97cc7d06af2c","Type":"ContainerStarted","Data":"2e56e19ff5eebba98c7e7900420446cc0e88abf2bcc97a482ed1b51c5ebc56da"}
Jul 24 15:52:44.950087 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:44.950036    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk" podStartSLOduration=1.781994643 podStartE2EDuration="1.950020516s" totalImagesPullingTime="168.025873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:44.948545022 +0000 UTC m=+706.500784796" watchObservedRunningTime="2026-07-24 15:52:44.950020516 +0000 UTC m=+706.502260288"
Jul 24 15:52:53.995439 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:53.995350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:52:54.163247 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:54.163213    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:54.163417 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:54.163271    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:54.175889 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:54.175860    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:52:54.989331 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:52:54.989303    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hvtwk"
Jul 24 15:53:03.579907 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:03.579868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:03.640958 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:03.640928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:41.729261 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:41.729227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:41.828376 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:41.828341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:46.419638 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:46.419595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:46.481258 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:46.481226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:53:52.462297 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:53:52.462257    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:17.588307 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:54:17.588271    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:17.656199 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:54:17.656163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:18.727881 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:54:18.727817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:18.830802 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:54:18.830754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:54:25.540506 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:54:25.540430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:55:07.186301 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.186260    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:55:07.186823 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.186490    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64456c74cb-rf87q" podUID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" containerName="manager" containerID="cri-o://33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8" gracePeriod=30
Jul 24 15:55:07.438106 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.438046    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:55:07.556823 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.556792    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert\") pod \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") "
Jul 24 15:55:07.557005 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.556872    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t44r7\" (UniqueName: \"kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7\") pod \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\" (UID: \"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9\") "
Jul 24 15:55:07.559139 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.559093    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7" (OuterVolumeSpecName: "kube-api-access-t44r7") pod "fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" (UID: "fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9"). InnerVolumeSpecName "kube-api-access-t44r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:55:07.559283 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.559144    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" (UID: "fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:55:07.565100 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.565072    2567 generic.go:356] "Generic (PLEG): container finished" podID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" containerID="33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8" exitCode=0
Jul 24 15:55:07.565253 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.565163    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-rf87q"
Jul 24 15:55:07.565253 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.565165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-rf87q" event={"ID":"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9","Type":"ContainerDied","Data":"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"}
Jul 24 15:55:07.565347 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.565266    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-rf87q" event={"ID":"fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9","Type":"ContainerDied","Data":"b9dba89520fbb9c3e9e85c9a677a1cb1b173c3b087564f43afc879242d09b8e1"}
Jul 24 15:55:07.565347 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.565292    2567 scope.go:117] "RemoveContainer" containerID="33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"
Jul 24 15:55:07.578037 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.578019    2567 scope.go:117] "RemoveContainer" containerID="33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"
Jul 24 15:55:07.578364 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:55:07.578345    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8\": container with ID starting with 33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8 not found: ID does not exist" containerID="33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"
Jul 24 15:55:07.578461 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.578373    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8"} err="failed to get container status \"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8\": rpc error: code = NotFound desc = could not find container \"33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8\": container with ID starting with 33dc4a88afe18fd7b5eca190bab015ca606d61f4be117990903770a03b859cd8 not found: ID does not exist"
Jul 24 15:55:07.587626 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.587601    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:55:07.591195 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.591167    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64456c74cb-rf87q"]
Jul 24 15:55:07.657847 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.657806    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t44r7\" (UniqueName: \"kubernetes.io/projected/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-kube-api-access-t44r7\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:55:07.657847 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:07.657841    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9-webhook-cert\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:55:08.948705 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:55:08.948665    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" path="/var/lib/kubelet/pods/fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9/volumes"
Jul 24 15:56:08.392796 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.392760    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:08.393257 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.393077    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" containerName="manager"
Jul 24 15:56:08.393257 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.393087    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" containerName="manager"
Jul 24 15:56:08.393257 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.393165    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbfe1e00-27ff-45fe-b0ef-8404ef69a7a9" containerName="manager"
Jul 24 15:56:08.397438 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.397420    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.400198 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.400170    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:56:08.400381 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.400361    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:56:08.401349 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.401329    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdgjn\""
Jul 24 15:56:08.408099 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.408080    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:08.502528 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.502489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fj4\" (UniqueName: \"kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.502701 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.502546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.603206 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.603170    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fj4\" (UniqueName: \"kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.603381 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.603232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.605865 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.605845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.612495 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.612473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fj4\" (UniqueName: \"kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4\") pod \"maas-controller-64456c74cb-dsfqp\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") " pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.707794 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.707704    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:08.849413 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:08.849389    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:08.852020 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:56:08.851982    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2646f3d2_1c54_41da_a63f_239f2b23154a.slice/crio-cde9a431fdb01d18c35f13692249c3cc79c3c7e38a0f0ca08fc42e2b57db0402 WatchSource:0}: Error finding container cde9a431fdb01d18c35f13692249c3cc79c3c7e38a0f0ca08fc42e2b57db0402: Status 404 returned error can't find the container with id cde9a431fdb01d18c35f13692249c3cc79c3c7e38a0f0ca08fc42e2b57db0402
Jul 24 15:56:09.842337 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:09.842296    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-dsfqp" event={"ID":"2646f3d2-1c54-41da-a63f-239f2b23154a","Type":"ContainerStarted","Data":"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"}
Jul 24 15:56:09.842773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:09.842390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-dsfqp" event={"ID":"2646f3d2-1c54-41da-a63f-239f2b23154a","Type":"ContainerStarted","Data":"cde9a431fdb01d18c35f13692249c3cc79c3c7e38a0f0ca08fc42e2b57db0402"}
Jul 24 15:56:09.842773 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:09.842453    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:09.864249 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:09.864193    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64456c74cb-dsfqp" podStartSLOduration=1.335883718 podStartE2EDuration="1.864175435s" totalImagesPullingTime="528.291717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:56:09.862316225 +0000 UTC m=+911.414555998" watchObservedRunningTime="2026-07-24 15:56:09.864175435 +0000 UTC m=+911.416415208"
Jul 24 15:56:20.849401 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:20.849370    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:25.410118 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.410080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:25.482353 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.482310    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:25.482560 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.482519    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-64456c74cb-dsfqp" podUID="2646f3d2-1c54-41da-a63f-239f2b23154a" containerName="manager" containerID="cri-o://4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5" gracePeriod=30
Jul 24 15:56:25.733858 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.733823    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:25.849675 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.849637    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4fj4\" (UniqueName: \"kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4\") pod \"2646f3d2-1c54-41da-a63f-239f2b23154a\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") "
Jul 24 15:56:25.849875 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.849736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert\") pod \"2646f3d2-1c54-41da-a63f-239f2b23154a\" (UID: \"2646f3d2-1c54-41da-a63f-239f2b23154a\") "
Jul 24 15:56:25.852228 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.852180    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4" (OuterVolumeSpecName: "kube-api-access-k4fj4") pod "2646f3d2-1c54-41da-a63f-239f2b23154a" (UID: "2646f3d2-1c54-41da-a63f-239f2b23154a"). InnerVolumeSpecName "kube-api-access-k4fj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:56:25.852411 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.852387    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2646f3d2-1c54-41da-a63f-239f2b23154a" (UID: "2646f3d2-1c54-41da-a63f-239f2b23154a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:56:25.912119 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.912078    2567 generic.go:356] "Generic (PLEG): container finished" podID="2646f3d2-1c54-41da-a63f-239f2b23154a" containerID="4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5" exitCode=0
Jul 24 15:56:25.912119 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.912136    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-dsfqp"
Jul 24 15:56:25.912340 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.912163    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-dsfqp" event={"ID":"2646f3d2-1c54-41da-a63f-239f2b23154a","Type":"ContainerDied","Data":"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"}
Jul 24 15:56:25.912340 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.912223    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-dsfqp" event={"ID":"2646f3d2-1c54-41da-a63f-239f2b23154a","Type":"ContainerDied","Data":"cde9a431fdb01d18c35f13692249c3cc79c3c7e38a0f0ca08fc42e2b57db0402"}
Jul 24 15:56:25.912340 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.912250    2567 scope.go:117] "RemoveContainer" containerID="4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"
Jul 24 15:56:25.921526 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.921501    2567 scope.go:117] "RemoveContainer" containerID="4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"
Jul 24 15:56:25.921847 ip-10-0-133-134 kubenswrapper[2567]: E0724 15:56:25.921826    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5\": container with ID starting with 4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5 not found: ID does not exist" containerID="4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"
Jul 24 15:56:25.921917 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.921860    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5"} err="failed to get container status \"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5\": rpc error: code = NotFound desc = could not find container \"4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5\": container with ID starting with 4f45addfda2b38498f4b6bcc6ef704aa925be1713a628caaa1a38d5adf2e3bb5 not found: ID does not exist"
Jul 24 15:56:25.934524 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.934483    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:25.936700 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.936669    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-64456c74cb-dsfqp"]
Jul 24 15:56:25.950803 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.950766    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4fj4\" (UniqueName: \"kubernetes.io/projected/2646f3d2-1c54-41da-a63f-239f2b23154a-kube-api-access-k4fj4\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:56:25.950803 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:25.950801    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2646f3d2-1c54-41da-a63f-239f2b23154a-webhook-cert\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 15:56:26.639400 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.639364    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:56:26.720604 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.720567    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64456c74cb-h6df5"]
Jul 24 15:56:26.720902 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.720889    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2646f3d2-1c54-41da-a63f-239f2b23154a" containerName="manager"
Jul 24 15:56:26.720948 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.720904    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2646f3d2-1c54-41da-a63f-239f2b23154a" containerName="manager"
Jul 24 15:56:26.720981 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.720966    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2646f3d2-1c54-41da-a63f-239f2b23154a" containerName="manager"
Jul 24 15:56:26.725367 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.725342    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.728223 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.728195    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:56:26.728223 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.728224    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:56:26.728456 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.728195    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdgjn\""
Jul 24 15:56:26.733358 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.733333    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-h6df5"]
Jul 24 15:56:26.759221 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.759186    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e0f3706-29cc-46a8-81fb-7b57d386114e-webhook-cert\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.759435 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.759337    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-896l7\" (UniqueName: \"kubernetes.io/projected/0e0f3706-29cc-46a8-81fb-7b57d386114e-kube-api-access-896l7\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.860463 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.860415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-896l7\" (UniqueName: \"kubernetes.io/projected/0e0f3706-29cc-46a8-81fb-7b57d386114e-kube-api-access-896l7\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.860632 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.860482    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e0f3706-29cc-46a8-81fb-7b57d386114e-webhook-cert\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.863245 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.863224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e0f3706-29cc-46a8-81fb-7b57d386114e-webhook-cert\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.869756 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.869729    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-896l7\" (UniqueName: \"kubernetes.io/projected/0e0f3706-29cc-46a8-81fb-7b57d386114e-kube-api-access-896l7\") pod \"maas-controller-64456c74cb-h6df5\" (UID: \"0e0f3706-29cc-46a8-81fb-7b57d386114e\") " pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:26.949239 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:26.949148    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2646f3d2-1c54-41da-a63f-239f2b23154a" path="/var/lib/kubelet/pods/2646f3d2-1c54-41da-a63f-239f2b23154a/volumes"
Jul 24 15:56:27.036578 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.036528    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:27.179103 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.179052    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64456c74cb-h6df5"]
Jul 24 15:56:27.181562 ip-10-0-133-134 kubenswrapper[2567]: W0724 15:56:27.181532    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e0f3706_29cc_46a8_81fb_7b57d386114e.slice/crio-b34f284d93a25f3129ee1aabdd0c96c428ae229e5382e42875ea909876d30596 WatchSource:0}: Error finding container b34f284d93a25f3129ee1aabdd0c96c428ae229e5382e42875ea909876d30596: Status 404 returned error can't find the container with id b34f284d93a25f3129ee1aabdd0c96c428ae229e5382e42875ea909876d30596
Jul 24 15:56:27.924529 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.924488    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-h6df5" event={"ID":"0e0f3706-29cc-46a8-81fb-7b57d386114e","Type":"ContainerStarted","Data":"e767bb29b3faaa9c0b6f4ecae2bd613f8646b965a665521a4b4677331ef029d5"}
Jul 24 15:56:27.924945 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.924546    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64456c74cb-h6df5" event={"ID":"0e0f3706-29cc-46a8-81fb-7b57d386114e","Type":"ContainerStarted","Data":"b34f284d93a25f3129ee1aabdd0c96c428ae229e5382e42875ea909876d30596"}
Jul 24 15:56:27.924945 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.924569    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:56:27.943841 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:27.943791    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64456c74cb-h6df5" podStartSLOduration=1.417386621 podStartE2EDuration="1.943773745s" totalImagesPullingTime="526.387124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:56:27.942482942 +0000 UTC m=+929.494722726" watchObservedRunningTime="2026-07-24 15:56:27.943773745 +0000 UTC m=+929.496013517"
Jul 24 15:56:38.930998 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:56:38.930964    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64456c74cb-h6df5"
Jul 24 15:57:01.273254 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:01.273218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:01.325967 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:01.325918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:02.469921 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:02.469881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:02.568087 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:02.568045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:02.695786 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:02.695748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:08.292885 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:08.292851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:08.398417 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:08.398383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:08.511463 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:08.511428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:08.665562 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:08.665482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:14.398976 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:14.398936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 15:57:14.460690 ip-10-0-133-134 kubenswrapper[2567]: I0724 15:57:14.460659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:00:02.087138 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:00:02.087099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:00:02.325048 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:00:02.325014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:00:30.391659 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:00:30.391549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:01:19.662971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:01:19.662935    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:25.655244 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:02:25.655207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:50.053448 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:02:50.053415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:02:50.154465 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:02:50.154436    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:04:45.988596 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:04:45.988562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:04:51.889509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:04:51.889423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:05:27.708791 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:05:27.708750    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:05:28.675839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:05:28.675804    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:07:05.888575 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:07:05.888539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:08:35.593290 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:08:35.593243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:08:57.080967 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:08:57.080929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:00.202023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:00.201987    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:00.304011 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:00.303978    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:00.649974 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:00.649939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:06.363659 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:06.363621    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:06.464815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:06.464782    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:06.616208 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:06.616105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:06.914354 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:06.914266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:07.426203 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:07.426165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:08.987717 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:08.987677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.085328 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.085293    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:09.088229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.088195    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.088955 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.088931    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.091704 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.091684    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a42991b8-serving-cert\""
Jul 24 16:09:09.100643 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.100600    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:09.154330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.154290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.154537 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.154343    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.154537 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.154401    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxprp\" (UniqueName: \"kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.179383 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.179333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.254887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.254783    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.254887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.254841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.255164 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.254903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxprp\" (UniqueName: \"kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.255719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.255694    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.257948 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.257923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.265068 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.265035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxprp\" (UniqueName: \"kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp\") pod \"maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.295370 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.295330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.401511 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.401478    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:09.562380 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.562304    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:09.565443 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:09.565411    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda026c1fa_849c_4724_b65e_5df602987b6b.slice/crio-52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f WatchSource:0}: Error finding container 52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f: Status 404 returned error can't find the container with id 52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f
Jul 24 16:09:09.566938 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.566915    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:09:09.760037 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.759970    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.861308 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.861226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:09.861997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.861975    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:09.866276 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.866262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:09.869626 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.869600    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a42991b8-dockercfg-jlhng\""
Jul 24 16:09:09.869760 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.869663    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a42991b8\""
Jul 24 16:09:09.877425 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.877397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:09.889185 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.889143    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:09.892544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.892518    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:09.914330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.914297    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:09.960659 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.960615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:09.961057 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:09.960780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7n5\" (UniqueName: \"kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.061607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.061575    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7n5\" (UniqueName: \"kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.062004 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.061684    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.062004 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.061732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.062004 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.061806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhzkt\" (UniqueName: \"kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.062503 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.062479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.070044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.070019    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7n5\" (UniqueName: \"kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5\") pod \"payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") " pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.162521 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.162420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.162521 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.162479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhzkt\" (UniqueName: \"kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.163268 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.163248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.171409 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.171380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhzkt\" (UniqueName: \"kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt\") pod \"payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.178213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.178120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:10.204489 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.204457    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:10.279448 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.279044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" event={"ID":"a026c1fa-849c-4724-b65e-5df602987b6b","Type":"ContainerStarted","Data":"6dfeb981d1d3edefc13555f0ea77969fe52f6fe09c766ab87497e303c47d31b6"}
Jul 24 16:09:10.279448 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.279105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" event={"ID":"a026c1fa-849c-4724-b65e-5df602987b6b","Type":"ContainerStarted","Data":"52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f"}
Jul 24 16:09:10.279448 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.279370    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:10.300149 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.300052    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" podStartSLOduration=0.822744398 podStartE2EDuration="1.300024903s" totalImagesPullingTime="477.280505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:10.298189243 +0000 UTC m=+1691.850429017" watchObservedRunningTime="2026-07-24 16:09:10.300024903 +0000 UTC m=+1691.852264677"
Jul 24 16:09:10.338272 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.338159    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:10.341841 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:10.341806    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc565c430_f150_4969_b36e_9085c31c012c.slice/crio-1489b821e35d45cea7e730e7f6ca54bfb7d9baf23c89084c5a1412ebc29c23f8 WatchSource:0}: Error finding container 1489b821e35d45cea7e730e7f6ca54bfb7d9baf23c89084c5a1412ebc29c23f8: Status 404 returned error can't find the container with id 1489b821e35d45cea7e730e7f6ca54bfb7d9baf23c89084c5a1412ebc29c23f8
Jul 24 16:09:10.370698 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:10.370652    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:10.380472 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:10.380439    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod460d30cc_e41d_4704_bdff_60d0d25b2ca0.slice/crio-91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292 WatchSource:0}: Error finding container 91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292: Status 404 returned error can't find the container with id 91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292
Jul 24 16:09:11.284965 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.284927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" event={"ID":"c565c430-f150-4969-b36e-9085c31c012c","Type":"ContainerStarted","Data":"709cd823dd3756cc96e352b048209430038f0500c248ceb41386ab441dd348ae"}
Jul 24 16:09:11.285471 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.284989    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" event={"ID":"c565c430-f150-4969-b36e-9085c31c012c","Type":"ContainerStarted","Data":"1489b821e35d45cea7e730e7f6ca54bfb7d9baf23c89084c5a1412ebc29c23f8"}
Jul 24 16:09:11.285471 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.285023    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:11.286652 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.286621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" event={"ID":"460d30cc-e41d-4704-bdff-60d0d25b2ca0","Type":"ContainerStarted","Data":"4af519f43555ee0a739c83ac0f8a6f558502ca689ee5b7cb07bbec92fa9909c6"}
Jul 24 16:09:11.286777 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.286673    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" event={"ID":"460d30cc-e41d-4704-bdff-60d0d25b2ca0","Type":"ContainerStarted","Data":"91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292"}
Jul 24 16:09:11.286821 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.286789    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:11.302469 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.302425    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" podStartSLOduration=2.302409346 podStartE2EDuration="2.302409346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:11.300370452 +0000 UTC m=+1692.852610225" watchObservedRunningTime="2026-07-24 16:09:11.302409346 +0000 UTC m=+1692.854649118"
Jul 24 16:09:11.323711 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:11.323657    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" podStartSLOduration=2.323637572 podStartE2EDuration="2.323637572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:11.322417573 +0000 UTC m=+1692.874657345" watchObservedRunningTime="2026-07-24 16:09:11.323637572 +0000 UTC m=+1692.875877344"
Jul 24 16:09:16.293354 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:16.293321    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:22.293940 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:22.293851    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:22.294385 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:22.294172    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:29.489338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:29.489303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:29.546044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:29.546005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:45.105766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.105723    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:45.106450 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.105975    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" podUID="a026c1fa-849c-4724-b65e-5df602987b6b" containerName="maas-api" containerID="cri-o://6dfeb981d1d3edefc13555f0ea77969fe52f6fe09c766ab87497e303c47d31b6" gracePeriod=30
Jul 24 16:09:45.133004 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.132976    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:45.163300 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.163263    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:45.163522 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.163495    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" podUID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" containerName="payload-processing" containerID="cri-o://4af519f43555ee0a739c83ac0f8a6f558502ca689ee5b7cb07bbec92fa9909c6" gracePeriod=30
Jul 24 16:09:45.170771 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.170736    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:45.170998 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.170973    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" podUID="c565c430-f150-4969-b36e-9085c31c012c" containerName="payload-pre-processing" containerID="cri-o://709cd823dd3756cc96e352b048209430038f0500c248ceb41386ab441dd348ae" gracePeriod=30
Jul 24 16:09:45.209421 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.209389    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:45.268536 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.268493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:45.443600 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.443569    2567 generic.go:356] "Generic (PLEG): container finished" podID="a026c1fa-849c-4724-b65e-5df602987b6b" containerID="6dfeb981d1d3edefc13555f0ea77969fe52f6fe09c766ab87497e303c47d31b6" exitCode=0
Jul 24 16:09:45.443764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.443654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" event={"ID":"a026c1fa-849c-4724-b65e-5df602987b6b","Type":"ContainerDied","Data":"6dfeb981d1d3edefc13555f0ea77969fe52f6fe09c766ab87497e303c47d31b6"}
Jul 24 16:09:45.443764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.443721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5" event={"ID":"a026c1fa-849c-4724-b65e-5df602987b6b","Type":"ContainerDied","Data":"52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f"}
Jul 24 16:09:45.443764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.443746    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52db8fa0bd805c42bfa52f0652b6a93e02569be1690664e435a5cb52f698668f"
Jul 24 16:09:45.445589 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.445563    2567 generic.go:356] "Generic (PLEG): container finished" podID="c565c430-f150-4969-b36e-9085c31c012c" containerID="709cd823dd3756cc96e352b048209430038f0500c248ceb41386ab441dd348ae" exitCode=0
Jul 24 16:09:45.445750 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.445593    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" event={"ID":"c565c430-f150-4969-b36e-9085c31c012c","Type":"ContainerDied","Data":"709cd823dd3756cc96e352b048209430038f0500c248ceb41386ab441dd348ae"}
Jul 24 16:09:45.447043 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.447006    2567 generic.go:356] "Generic (PLEG): container finished" podID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" containerID="4af519f43555ee0a739c83ac0f8a6f558502ca689ee5b7cb07bbec92fa9909c6" exitCode=0
Jul 24 16:09:45.447043 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.447039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" event={"ID":"460d30cc-e41d-4704-bdff-60d0d25b2ca0","Type":"ContainerDied","Data":"4af519f43555ee0a739c83ac0f8a6f558502ca689ee5b7cb07bbec92fa9909c6"}
Jul 24 16:09:45.447223 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.447080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw" event={"ID":"460d30cc-e41d-4704-bdff-60d0d25b2ca0","Type":"ContainerDied","Data":"91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292"}
Jul 24 16:09:45.447223 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.447097    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91363ea1a3020fee9d5e3345b3f861437e0fd58da47d78cb4fe34e4c78bb1292"
Jul 24 16:09:45.447837 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.447818    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:45.451385 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.451361    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:45.475295 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.475270    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:45.554478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554437    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca\") pod \"a026c1fa-849c-4724-b65e-5df602987b6b\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") "
Jul 24 16:09:45.554478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554477    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm7n5\" (UniqueName: \"kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5\") pod \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") "
Jul 24 16:09:45.554744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554507    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhzkt\" (UniqueName: \"kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt\") pod \"c565c430-f150-4969-b36e-9085c31c012c\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") "
Jul 24 16:09:45.554744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554528    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls\") pod \"a026c1fa-849c-4724-b65e-5df602987b6b\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") "
Jul 24 16:09:45.554744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume\") pod \"c565c430-f150-4969-b36e-9085c31c012c\" (UID: \"c565c430-f150-4969-b36e-9085c31c012c\") "
Jul 24 16:09:45.554744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume\") pod \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\" (UID: \"460d30cc-e41d-4704-bdff-60d0d25b2ca0\") "
Jul 24 16:09:45.554744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554644    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxprp\" (UniqueName: \"kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp\") pod \"a026c1fa-849c-4724-b65e-5df602987b6b\" (UID: \"a026c1fa-849c-4724-b65e-5df602987b6b\") "
Jul 24 16:09:45.555007 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.554967    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a026c1fa-849c-4724-b65e-5df602987b6b" (UID: "a026c1fa-849c-4724-b65e-5df602987b6b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:09:45.555063 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.555033    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c565c430-f150-4969-b36e-9085c31c012c" (UID: "c565c430-f150-4969-b36e-9085c31c012c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:09:45.555121 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.555078    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "460d30cc-e41d-4704-bdff-60d0d25b2ca0" (UID: "460d30cc-e41d-4704-bdff-60d0d25b2ca0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:09:45.556994 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.556963    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt" (OuterVolumeSpecName: "kube-api-access-qhzkt") pod "c565c430-f150-4969-b36e-9085c31c012c" (UID: "c565c430-f150-4969-b36e-9085c31c012c"). InnerVolumeSpecName "kube-api-access-qhzkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:09:45.557118 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.557013    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp" (OuterVolumeSpecName: "kube-api-access-kxprp") pod "a026c1fa-849c-4724-b65e-5df602987b6b" (UID: "a026c1fa-849c-4724-b65e-5df602987b6b"). InnerVolumeSpecName "kube-api-access-kxprp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:09:45.557193 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.557178    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a026c1fa-849c-4724-b65e-5df602987b6b" (UID: "a026c1fa-849c-4724-b65e-5df602987b6b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:09:45.557255 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.557219    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5" (OuterVolumeSpecName: "kube-api-access-qm7n5") pod "460d30cc-e41d-4704-bdff-60d0d25b2ca0" (UID: "460d30cc-e41d-4704-bdff-60d0d25b2ca0"). InnerVolumeSpecName "kube-api-access-qm7n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655667    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a026c1fa-849c-4724-b65e-5df602987b6b-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655701    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm7n5\" (UniqueName: \"kubernetes.io/projected/460d30cc-e41d-4704-bdff-60d0d25b2ca0-kube-api-access-qm7n5\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655714    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhzkt\" (UniqueName: \"kubernetes.io/projected/c565c430-f150-4969-b36e-9085c31c012c-kube-api-access-qhzkt\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655727    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a026c1fa-849c-4724-b65e-5df602987b6b-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655739    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c565c430-f150-4969-b36e-9085c31c012c-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655751    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/460d30cc-e41d-4704-bdff-60d0d25b2ca0-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:45.655766 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:45.655763    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxprp\" (UniqueName: \"kubernetes.io/projected/a026c1fa-849c-4724-b65e-5df602987b6b-kube-api-access-kxprp\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:09:46.452547 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.452457    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"
Jul 24 16:09:46.452547 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.452499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r" event={"ID":"c565c430-f150-4969-b36e-9085c31c012c","Type":"ContainerDied","Data":"1489b821e35d45cea7e730e7f6ca54bfb7d9baf23c89084c5a1412ebc29c23f8"}
Jul 24 16:09:46.453031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.452568    2567 scope.go:117] "RemoveContainer" containerID="709cd823dd3756cc96e352b048209430038f0500c248ceb41386ab441dd348ae"
Jul 24 16:09:46.453031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.452725    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"
Jul 24 16:09:46.453031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.452914    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"
Jul 24 16:09:46.480494 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.480462    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:46.484609 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.484581    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a42991b8-9c96b6fc6-5jw7r"]
Jul 24 16:09:46.496213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.496185    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:46.500736 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.500706    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a42991b8-b7c5cc49b-7slcw"]
Jul 24 16:09:46.512807 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.512772    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:46.516302 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.516278    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a42991b8-6bd7cb64fb-rg2k5"]
Jul 24 16:09:46.949414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.949375    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" path="/var/lib/kubelet/pods/460d30cc-e41d-4704-bdff-60d0d25b2ca0/volumes"
Jul 24 16:09:46.949832 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.949811    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a026c1fa-849c-4724-b65e-5df602987b6b" path="/var/lib/kubelet/pods/a026c1fa-849c-4724-b65e-5df602987b6b/volumes"
Jul 24 16:09:46.950206 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:46.950189    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c565c430-f150-4969-b36e-9085c31c012c" path="/var/lib/kubelet/pods/c565c430-f150-4969-b36e-9085c31c012c/volumes"
Jul 24 16:09:49.576661 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:49.576629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:49.727600 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:49.727566    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:49.960775 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:49.960687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:50.246478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:50.246430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:50.343859 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:50.343821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:50.447346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:50.447305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:50.805849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:50.805801    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:51.125560 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:51.125461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:53.271578 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.271547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:53.351576 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.351539    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:09:53.352064 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352047    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a026c1fa-849c-4724-b65e-5df602987b6b" containerName="maas-api"
Jul 24 16:09:53.352109 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352067    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a026c1fa-849c-4724-b65e-5df602987b6b" containerName="maas-api"
Jul 24 16:09:53.352109 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352080    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c565c430-f150-4969-b36e-9085c31c012c" containerName="payload-pre-processing"
Jul 24 16:09:53.352109 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352089    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c565c430-f150-4969-b36e-9085c31c012c" containerName="payload-pre-processing"
Jul 24 16:09:53.352109 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352104    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" containerName="payload-processing"
Jul 24 16:09:53.352269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352114    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" containerName="payload-processing"
Jul 24 16:09:53.352269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352219    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="460d30cc-e41d-4704-bdff-60d0d25b2ca0" containerName="payload-processing"
Jul 24 16:09:53.352269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352231    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a026c1fa-849c-4724-b65e-5df602987b6b" containerName="maas-api"
Jul 24 16:09:53.352269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.352240    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c565c430-f150-4969-b36e-9085c31c012c" containerName="payload-pre-processing"
Jul 24 16:09:53.356751 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.356724    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.359060 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.359034    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-54f72705-serving-cert\""
Jul 24 16:09:53.367914 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.367883    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:09:53.391595 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.391564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:53.426671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.426637    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.426886 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.426694    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kgk8\" (UniqueName: \"kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.426886 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.426768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.528263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.528161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.528263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.528246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.528509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.528305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kgk8\" (UniqueName: \"kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.528509 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:09:53.528365    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-serving-cert: secret "maas-api-e2e-migrate-54f72705-serving-cert" not found
Jul 24 16:09:53.528509 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:09:53.528438    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls podName:6d63d675-543f-41c9-bfb6-e85c75d6b31b nodeName:}" failed. No retries permitted until 2026-07-24 16:09:54.028419412 +0000 UTC m=+1735.580659163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls") pod "maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" (UID: "6d63d675-543f-41c9-bfb6-e85c75d6b31b") : secret "maas-api-e2e-migrate-54f72705-serving-cert" not found
Jul 24 16:09:53.528828 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.528807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.537332 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.537302    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kgk8\" (UniqueName: \"kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:53.933660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:53.933576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:54.021586 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.021542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:54.033762 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.033725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:54.036599 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.036570    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") pod \"maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:54.100669 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.100626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:09:54.104260 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.104233    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:09:54.107781 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.107761    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.110536 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.110511    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-54f72705\""
Jul 24 16:09:54.110651 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.110575    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-54f72705-dockercfg-qgmld\""
Jul 24 16:09:54.121429 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.121396    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:09:54.133172 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.133106    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:09:54.136706 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.136676    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.155091 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.155055    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:09:54.235418 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.235383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.235617 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.235460    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v76k\" (UniqueName: \"kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.235617 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.235485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.235617 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.235535    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc9xw\" (UniqueName: \"kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.269479 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.269437    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:54.336937 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.336900    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc9xw\" (UniqueName: \"kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.337778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.336984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.337778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.337046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v76k\" (UniqueName: \"kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.337778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.337065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.337993 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.337914    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.338715 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.338688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.351402 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.351366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v76k\" (UniqueName: \"kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k\") pod \"payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") " pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.352211 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.352188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc9xw\" (UniqueName: \"kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw\") pod \"payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.418082 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.418054    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:09:54.419156 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.419111    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:54.420719 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:54.420690    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d63d675_543f_41c9_bfb6_e85c75d6b31b.slice/crio-f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe WatchSource:0}: Error finding container f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe: Status 404 returned error can't find the container with id f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe
Jul 24 16:09:54.447928 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.447900    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:54.493017 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.492928    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" event={"ID":"6d63d675-543f-41c9-bfb6-e85c75d6b31b","Type":"ContainerStarted","Data":"f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe"}
Jul 24 16:09:54.581393 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.581365    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:09:54.583918 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:54.583881    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257a9887_2960_40a3_be77_a4bf5c1d35a0.slice/crio-c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf WatchSource:0}: Error finding container c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf: Status 404 returned error can't find the container with id c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf
Jul 24 16:09:54.607943 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:54.607913    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:09:54.610575 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:09:54.610546    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1e7da3_aa37_496c_b6a8_58aa0577d9c8.slice/crio-ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde WatchSource:0}: Error finding container ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde: Status 404 returned error can't find the container with id ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde
Jul 24 16:09:55.499825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.499786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" event={"ID":"6d63d675-543f-41c9-bfb6-e85c75d6b31b","Type":"ContainerStarted","Data":"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"}
Jul 24 16:09:55.500344 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.499950    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:09:55.501508 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.501478    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" event={"ID":"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8","Type":"ContainerStarted","Data":"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"}
Jul 24 16:09:55.501643 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.501529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" event={"ID":"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8","Type":"ContainerStarted","Data":"ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde"}
Jul 24 16:09:55.501643 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.501561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:09:55.502954 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.502930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" event={"ID":"257a9887-2960-40a3-be77-a4bf5c1d35a0","Type":"ContainerStarted","Data":"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"}
Jul 24 16:09:55.503079 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.502973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" event={"ID":"257a9887-2960-40a3-be77-a4bf5c1d35a0","Type":"ContainerStarted","Data":"c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf"}
Jul 24 16:09:55.503079 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.503000    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:09:55.518018 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.517962    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" podStartSLOduration=2.151846846 podStartE2EDuration="2.517947212s" totalImagesPullingTime="366.100366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:55.515497489 +0000 UTC m=+1737.067737263" watchObservedRunningTime="2026-07-24 16:09:55.517947212 +0000 UTC m=+1737.070187039"
Jul 24 16:09:55.532077 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.532027    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" podStartSLOduration=1.532009697 podStartE2EDuration="1.532009697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:55.530437421 +0000 UTC m=+1737.082677204" watchObservedRunningTime="2026-07-24 16:09:55.532009697 +0000 UTC m=+1737.084249542"
Jul 24 16:09:55.548928 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:09:55.548878    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" podStartSLOduration=1.548862045 podStartE2EDuration="1.548862045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:55.546685063 +0000 UTC m=+1737.098924836" watchObservedRunningTime="2026-07-24 16:09:55.548862045 +0000 UTC m=+1737.101101817"
Jul 24 16:10:01.513515 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:01.513484    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:10:06.509876 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:06.509842    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:10:06.510369 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:06.510329    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:10:28.756799 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.756763    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:10:28.757406 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.757050    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" podUID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" containerName="maas-api" containerID="cri-o://8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3" gracePeriod=30
Jul 24 16:10:28.781121 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.781078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:28.805693 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.805660    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:10:28.805987 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.805900    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" podUID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" containerName="payload-processing" containerID="cri-o://c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c" gracePeriod=30
Jul 24 16:10:28.813741 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.813691    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:10:28.814064 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.813885    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" podUID="257a9887-2960-40a3-be77-a4bf5c1d35a0" containerName="payload-pre-processing" containerID="cri-o://adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615" gracePeriod=30
Jul 24 16:10:28.876548 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.876505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:28.976734 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:28.976701    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:29.078620 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.078596    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:10:29.083834 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.083804    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:10:29.117685 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.117656    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:10:29.143156 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143109    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca\") pod \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") "
Jul 24 16:10:29.143330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143196    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") pod \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") "
Jul 24 16:10:29.143330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143221    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kgk8\" (UniqueName: \"kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8\") pod \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\" (UID: \"6d63d675-543f-41c9-bfb6-e85c75d6b31b\") "
Jul 24 16:10:29.143330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143239    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v76k\" (UniqueName: \"kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k\") pod \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") "
Jul 24 16:10:29.143330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume\") pod \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\" (UID: \"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8\") "
Jul 24 16:10:29.143551 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143504    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d63d675-543f-41c9-bfb6-e85c75d6b31b" (UID: "6d63d675-543f-41c9-bfb6-e85c75d6b31b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:10:29.143747 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.143724    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" (UID: "ac1e7da3-aa37-496c-b6a8-58aa0577d9c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:10:29.145575 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.145549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8" (OuterVolumeSpecName: "kube-api-access-9kgk8") pod "6d63d675-543f-41c9-bfb6-e85c75d6b31b" (UID: "6d63d675-543f-41c9-bfb6-e85c75d6b31b"). InnerVolumeSpecName "kube-api-access-9kgk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:10:29.145689 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.145668    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k" (OuterVolumeSpecName: "kube-api-access-5v76k") pod "ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" (UID: "ac1e7da3-aa37-496c-b6a8-58aa0577d9c8"). InnerVolumeSpecName "kube-api-access-5v76k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:10:29.145942 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.145928    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d63d675-543f-41c9-bfb6-e85c75d6b31b" (UID: "6d63d675-543f-41c9-bfb6-e85c75d6b31b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:10:29.244470 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume\") pod \"257a9887-2960-40a3-be77-a4bf5c1d35a0\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") "
Jul 24 16:10:29.244649 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244525    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc9xw\" (UniqueName: \"kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw\") pod \"257a9887-2960-40a3-be77-a4bf5c1d35a0\" (UID: \"257a9887-2960-40a3-be77-a4bf5c1d35a0\") "
Jul 24 16:10:29.244751 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244738    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d63d675-543f-41c9-bfb6-e85c75d6b31b-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.244789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244755    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d63d675-543f-41c9-bfb6-e85c75d6b31b-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.244789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244764    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kgk8\" (UniqueName: \"kubernetes.io/projected/6d63d675-543f-41c9-bfb6-e85c75d6b31b-kube-api-access-9kgk8\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.244789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244773    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v76k\" (UniqueName: \"kubernetes.io/projected/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-kube-api-access-5v76k\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.244789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244782    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.244910 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.244787    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "257a9887-2960-40a3-be77-a4bf5c1d35a0" (UID: "257a9887-2960-40a3-be77-a4bf5c1d35a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:10:29.246700 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.246678    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw" (OuterVolumeSpecName: "kube-api-access-vc9xw") pod "257a9887-2960-40a3-be77-a4bf5c1d35a0" (UID: "257a9887-2960-40a3-be77-a4bf5c1d35a0"). InnerVolumeSpecName "kube-api-access-vc9xw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:10:29.345637 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.345528    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc9xw\" (UniqueName: \"kubernetes.io/projected/257a9887-2960-40a3-be77-a4bf5c1d35a0-kube-api-access-vc9xw\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.345637 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.345568    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/257a9887-2960-40a3-be77-a4bf5c1d35a0-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:10:29.665282 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.665192    2567 generic.go:356] "Generic (PLEG): container finished" podID="257a9887-2960-40a3-be77-a4bf5c1d35a0" containerID="adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615" exitCode=0
Jul 24 16:10:29.665282 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.665225    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"
Jul 24 16:10:29.665518 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.665219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" event={"ID":"257a9887-2960-40a3-be77-a4bf5c1d35a0","Type":"ContainerDied","Data":"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"}
Jul 24 16:10:29.665518 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.665397    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd" event={"ID":"257a9887-2960-40a3-be77-a4bf5c1d35a0","Type":"ContainerDied","Data":"c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf"}
Jul 24 16:10:29.665518 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.665432    2567 scope.go:117] "RemoveContainer" containerID="adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"
Jul 24 16:10:29.666661 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.666642    2567 generic.go:356] "Generic (PLEG): container finished" podID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" containerID="8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3" exitCode=0
Jul 24 16:10:29.666767 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.666692    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"
Jul 24 16:10:29.666824 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.666780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" event={"ID":"6d63d675-543f-41c9-bfb6-e85c75d6b31b","Type":"ContainerDied","Data":"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"}
Jul 24 16:10:29.666868 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.666831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9" event={"ID":"6d63d675-543f-41c9-bfb6-e85c75d6b31b","Type":"ContainerDied","Data":"f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe"}
Jul 24 16:10:29.673078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.673004    2567 generic.go:356] "Generic (PLEG): container finished" podID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" containerID="c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c" exitCode=0
Jul 24 16:10:29.673078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.673059    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" event={"ID":"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8","Type":"ContainerDied","Data":"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"}
Jul 24 16:10:29.673078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.673066    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"
Jul 24 16:10:29.673354 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.673105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql" event={"ID":"ac1e7da3-aa37-496c-b6a8-58aa0577d9c8","Type":"ContainerDied","Data":"ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde"}
Jul 24 16:10:29.677618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.677600    2567 scope.go:117] "RemoveContainer" containerID="adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"
Jul 24 16:10:29.677925 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:29.677906    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615\": container with ID starting with adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615 not found: ID does not exist" containerID="adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"
Jul 24 16:10:29.678014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.677932    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615"} err="failed to get container status \"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615\": rpc error: code = NotFound desc = could not find container \"adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615\": container with ID starting with adcfc26917a63a1c1fddaecc62b9a6589aed68e31be6238061ac2ad6e7c1c615 not found: ID does not exist"
Jul 24 16:10:29.678014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.677954    2567 scope.go:117] "RemoveContainer" containerID="8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"
Jul 24 16:10:29.687939 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.687920    2567 scope.go:117] "RemoveContainer" containerID="8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"
Jul 24 16:10:29.688230 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:29.688212    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3\": container with ID starting with 8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3 not found: ID does not exist" containerID="8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"
Jul 24 16:10:29.688317 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.688239    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3"} err="failed to get container status \"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3\": rpc error: code = NotFound desc = could not find container \"8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3\": container with ID starting with 8b3cbfd326ede5b8c3d37301ddc07194f3b756761ae2900bf5e0d02495a753b3 not found: ID does not exist"
Jul 24 16:10:29.688317 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.688262    2567 scope.go:117] "RemoveContainer" containerID="c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"
Jul 24 16:10:29.694165 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.694120    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:10:29.698980 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.698951    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-54f72705-579f79cbf7-bdkr9"]
Jul 24 16:10:29.699744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.699728    2567 scope.go:117] "RemoveContainer" containerID="c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"
Jul 24 16:10:29.700080 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:29.700050    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c\": container with ID starting with c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c not found: ID does not exist" containerID="c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"
Jul 24 16:10:29.700150 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.700091    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c"} err="failed to get container status \"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c\": rpc error: code = NotFound desc = could not find container \"c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c\": container with ID starting with c4aeab351ec456846f58ef2966c4d0a210fb62565471de1363d5a663ad8eb80c not found: ID does not exist"
Jul 24 16:10:29.708022 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.707997    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:10:29.709936 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:29.709902    2567 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d63d675_543f_41c9_bfb6_e85c75d6b31b.slice/crio-f6e6f5b5bcd39ccf6fb6329238d341d712bcfaf9daad332ea61d5d1f9fa909fe\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257a9887_2960_40a3_be77_a4bf5c1d35a0.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d63d675_543f_41c9_bfb6_e85c75d6b31b.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1e7da3_aa37_496c_b6a8_58aa0577d9c8.slice/crio-ebbe598b56c0bbaa0ee7e601417ac197c2e79dc33fbf472de56ee699dcd5ddde\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257a9887_2960_40a3_be77_a4bf5c1d35a0.slice/crio-c2e148f2cfe0c3fb56f580f05ab2f9e2a697cf1f7aa8eaae946caa13514b5ebf\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1e7da3_aa37_496c_b6a8_58aa0577d9c8.slice\": RecentStats: unable to find data in memory cache]"
Jul 24 16:10:29.712252 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.712230    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-54f72705-67457ccd8d-2l4vd"]
Jul 24 16:10:29.723044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.723014    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:10:29.726426 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:29.726402    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-54f72705-77bbfbc597-f25ql"]
Jul 24 16:10:30.949880 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:30.949841    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="257a9887-2960-40a3-be77-a4bf5c1d35a0" path="/var/lib/kubelet/pods/257a9887-2960-40a3-be77-a4bf5c1d35a0/volumes"
Jul 24 16:10:30.950450 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:30.950427    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" path="/var/lib/kubelet/pods/6d63d675-543f-41c9-bfb6-e85c75d6b31b/volumes"
Jul 24 16:10:30.950808 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:30.950790    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" path="/var/lib/kubelet/pods/ac1e7da3-aa37-496c-b6a8-58aa0577d9c8/volumes"
Jul 24 16:10:33.377062 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:33.377012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:33.539229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:33.539183    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:33.812532 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:33.812491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:34.042928 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:34.042890    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:34.359336 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:34.359287    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:34.748008 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:34.747964    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:36.406667 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.406615    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:36.506803 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.506767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:36.675221 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675101    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:10:36.675475 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675460    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" containerName="payload-processing"
Jul 24 16:10:36.675544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675476    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" containerName="payload-processing"
Jul 24 16:10:36.675544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675485    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" containerName="maas-api"
Jul 24 16:10:36.675544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675492    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" containerName="maas-api"
Jul 24 16:10:36.675544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675501    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257a9887-2960-40a3-be77-a4bf5c1d35a0" containerName="payload-pre-processing"
Jul 24 16:10:36.675544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675507    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="257a9887-2960-40a3-be77-a4bf5c1d35a0" containerName="payload-pre-processing"
Jul 24 16:10:36.675801 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675564    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac1e7da3-aa37-496c-b6a8-58aa0577d9c8" containerName="payload-processing"
Jul 24 16:10:36.675801 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675572    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="257a9887-2960-40a3-be77-a4bf5c1d35a0" containerName="payload-pre-processing"
Jul 24 16:10:36.675801 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.675579    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d63d675-543f-41c9-bfb6-e85c75d6b31b" containerName="maas-api"
Jul 24 16:10:36.679741 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.679717    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.683314 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.683289    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e125f3d6-serving-cert\""
Jul 24 16:10:36.696203 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.696166    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:10:36.736117 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.736080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:36.813654 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.813611    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.813845 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.813686    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7kgq\" (UniqueName: \"kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.813845 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.813728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.914345 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.914304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.914534 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.914370    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.914534 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.914459    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7kgq\" (UniqueName: \"kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.914534 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:36.914475    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-serving-cert: secret "maas-api-e2e-ait-e125f3d6-serving-cert" not found
Jul 24 16:10:36.914656 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:10:36.914557    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls podName:d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe nodeName:}" failed. No retries permitted until 2026-07-24 16:10:37.41453914 +0000 UTC m=+1778.966778890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls") pod "maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" (UID: "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe") : secret "maas-api-e2e-ait-e125f3d6-serving-cert" not found
Jul 24 16:10:36.915003 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.914985    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:36.926369 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:36.926294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7kgq\" (UniqueName: \"kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:37.279020 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.278975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:37.419813 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.419780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:37.422549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.422523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") pod \"maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:37.427616 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.427588    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:37.528189 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.528154    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:10:37.590344 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.590263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:37.636534 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.636500    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:10:37.640146 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.640104    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.643922 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.642808    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e125f3d6\""
Jul 24 16:10:37.643922 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.643369    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e125f3d6-dockercfg-2pz8p\""
Jul 24 16:10:37.652161 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.652027    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:10:37.690212 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.690078    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:10:37.695604 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.695575    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.703182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.703150    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:10:37.722767 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.722735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52hft\" (UniqueName: \"kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.722960 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.722780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.752098 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.752035    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:10:37.755715 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:10:37.755676    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b107f0_b0bb_4bff_bd29_d7a9c1f0f7fe.slice/crio-b7b125bc5a7037f88f6e92660b57ac4753f92337fd0ed1b6a7ba40f516cc4d3c WatchSource:0}: Error finding container b7b125bc5a7037f88f6e92660b57ac4753f92337fd0ed1b6a7ba40f516cc4d3c: Status 404 returned error can't find the container with id b7b125bc5a7037f88f6e92660b57ac4753f92337fd0ed1b6a7ba40f516cc4d3c
Jul 24 16:10:37.823715 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.823680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.823907 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.823725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52hft\" (UniqueName: \"kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.823907 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.823753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.823907 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.823873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5hcx\" (UniqueName: \"kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.824482 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.824462    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.833185 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.833158    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52hft\" (UniqueName: \"kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft\") pod \"payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:37.924552 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.924462    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.924552 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.924539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5hcx\" (UniqueName: \"kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.925282 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.925250    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.938934 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.938908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5hcx\" (UniqueName: \"kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx\") pod \"payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") " pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:37.954953 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:37.954910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:38.014103 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.014073    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:38.104939 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.104909    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:10:38.106824 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:10:38.106793    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44732f64_6aa3_4ea2_b19e_ad2aa07a762e.slice/crio-e71033670dc42a5414f38836424a8cdf05638a17119745c3d72a42d740691370 WatchSource:0}: Error finding container e71033670dc42a5414f38836424a8cdf05638a17119745c3d72a42d740691370: Status 404 returned error can't find the container with id e71033670dc42a5414f38836424a8cdf05638a17119745c3d72a42d740691370
Jul 24 16:10:38.168406 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.168380    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:10:38.170553 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:10:38.170526    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46c2c4de_19f5_43f0_976b_d01ef900c9a0.slice/crio-9c05dd3a3d65108b32a83c8da350e0b5e58c96cf0134607a6a07c894706a6bd5 WatchSource:0}: Error finding container 9c05dd3a3d65108b32a83c8da350e0b5e58c96cf0134607a6a07c894706a6bd5: Status 404 returned error can't find the container with id 9c05dd3a3d65108b32a83c8da350e0b5e58c96cf0134607a6a07c894706a6bd5
Jul 24 16:10:38.722567 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.722520    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" event={"ID":"44732f64-6aa3-4ea2-b19e-ad2aa07a762e","Type":"ContainerStarted","Data":"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"}
Jul 24 16:10:38.723038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.722624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" event={"ID":"44732f64-6aa3-4ea2-b19e-ad2aa07a762e","Type":"ContainerStarted","Data":"e71033670dc42a5414f38836424a8cdf05638a17119745c3d72a42d740691370"}
Jul 24 16:10:38.723038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.722658    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:10:38.724195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.724167    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" event={"ID":"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe","Type":"ContainerStarted","Data":"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"}
Jul 24 16:10:38.724309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.724212    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" event={"ID":"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe","Type":"ContainerStarted","Data":"b7b125bc5a7037f88f6e92660b57ac4753f92337fd0ed1b6a7ba40f516cc4d3c"}
Jul 24 16:10:38.724309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.724247    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:38.725679 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.725659    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" event={"ID":"46c2c4de-19f5-43f0-976b-d01ef900c9a0","Type":"ContainerStarted","Data":"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"}
Jul 24 16:10:38.725789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.725694    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" event={"ID":"46c2c4de-19f5-43f0-976b-d01ef900c9a0","Type":"ContainerStarted","Data":"9c05dd3a3d65108b32a83c8da350e0b5e58c96cf0134607a6a07c894706a6bd5"}
Jul 24 16:10:38.725789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.725716    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:38.738763 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.738715    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" podStartSLOduration=1.738698206 podStartE2EDuration="1.738698206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:10:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:10:38.737535678 +0000 UTC m=+1780.289775447" watchObservedRunningTime="2026-07-24 16:10:38.738698206 +0000 UTC m=+1780.290937980"
Jul 24 16:10:38.756444 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.756390    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" podStartSLOduration=1.7563756449999999 podStartE2EDuration="1.756375645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:10:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:10:38.754325785 +0000 UTC m=+1780.306565558" watchObservedRunningTime="2026-07-24 16:10:38.756375645 +0000 UTC m=+1780.308615425"
Jul 24 16:10:38.770609 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:38.770549    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" podStartSLOduration=2.317365947 podStartE2EDuration="2.77052772s" totalImagesPullingTime="453.161773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:10:38.770099324 +0000 UTC m=+1780.322339087" watchObservedRunningTime="2026-07-24 16:10:38.77052772 +0000 UTC m=+1780.322767494"
Jul 24 16:10:44.737353 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:44.737323    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:10:49.733369 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:49.733284    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:10:49.733800 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:10:49.733469    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:11:12.795972 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.795934    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:11:12.796572 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.796224    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" podUID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" containerName="maas-api" containerID="cri-o://8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6" gracePeriod=30
Jul 24 16:11:12.819228 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.819195    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:12.847748 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.847708    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:11:12.848192 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.847965    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" podUID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" containerName="payload-processing" containerID="cri-o://6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470" gracePeriod=30
Jul 24 16:11:12.856703 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.856670    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:11:12.857054 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.857018    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" podUID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" containerName="payload-pre-processing" containerID="cri-o://8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769" gracePeriod=30
Jul 24 16:11:12.895209 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.895163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:12.997298 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:12.996955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:13.137371 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.137344    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:11:13.140274 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.140251    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:11:13.162117 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.162085    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:11:13.260591 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260550    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume\") pod \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") "
Jul 24 16:11:13.260591 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260595    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5hcx\" (UniqueName: \"kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx\") pod \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\" (UID: \"46c2c4de-19f5-43f0-976b-d01ef900c9a0\") "
Jul 24 16:11:13.260854 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260634    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7kgq\" (UniqueName: \"kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq\") pod \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") "
Jul 24 16:11:13.260854 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260683    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca\") pod \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") "
Jul 24 16:11:13.260854 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260708    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") pod \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\" (UID: \"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe\") "
Jul 24 16:11:13.260854 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260765    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume\") pod \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") "
Jul 24 16:11:13.260854 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.260804    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52hft\" (UniqueName: \"kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft\") pod \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\" (UID: \"44732f64-6aa3-4ea2-b19e-ad2aa07a762e\") "
Jul 24 16:11:13.261158 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.261050    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" (UID: "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:13.261227 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.261117    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46c2c4de-19f5-43f0-976b-d01ef900c9a0" (UID: "46c2c4de-19f5-43f0-976b-d01ef900c9a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:13.261280 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.261236    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44732f64-6aa3-4ea2-b19e-ad2aa07a762e" (UID: "44732f64-6aa3-4ea2-b19e-ad2aa07a762e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:13.263007 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.262976    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" (UID: "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:11:13.263007 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.262990    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx" (OuterVolumeSpecName: "kube-api-access-g5hcx") pod "46c2c4de-19f5-43f0-976b-d01ef900c9a0" (UID: "46c2c4de-19f5-43f0-976b-d01ef900c9a0"). InnerVolumeSpecName "kube-api-access-g5hcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:13.263236 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.263003    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq" (OuterVolumeSpecName: "kube-api-access-j7kgq") pod "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" (UID: "d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe"). InnerVolumeSpecName "kube-api-access-j7kgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:13.263308 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.263267    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft" (OuterVolumeSpecName: "kube-api-access-52hft") pod "44732f64-6aa3-4ea2-b19e-ad2aa07a762e" (UID: "44732f64-6aa3-4ea2-b19e-ad2aa07a762e"). InnerVolumeSpecName "kube-api-access-52hft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362406    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7kgq\" (UniqueName: \"kubernetes.io/projected/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-kube-api-access-j7kgq\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362439    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362450    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362459    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362467    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52hft\" (UniqueName: \"kubernetes.io/projected/44732f64-6aa3-4ea2-b19e-ad2aa07a762e-kube-api-access-52hft\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362477    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46c2c4de-19f5-43f0-976b-d01ef900c9a0-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.362499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.362485    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5hcx\" (UniqueName: \"kubernetes.io/projected/46c2c4de-19f5-43f0-976b-d01ef900c9a0-kube-api-access-g5hcx\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:13.892963 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.892926    2567 generic.go:356] "Generic (PLEG): container finished" podID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" containerID="8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769" exitCode=0
Jul 24 16:11:13.893458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.892987    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"
Jul 24 16:11:13.893458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.893011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" event={"ID":"44732f64-6aa3-4ea2-b19e-ad2aa07a762e","Type":"ContainerDied","Data":"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"}
Jul 24 16:11:13.893458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.893149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq" event={"ID":"44732f64-6aa3-4ea2-b19e-ad2aa07a762e","Type":"ContainerDied","Data":"e71033670dc42a5414f38836424a8cdf05638a17119745c3d72a42d740691370"}
Jul 24 16:11:13.893458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.893187    2567 scope.go:117] "RemoveContainer" containerID="8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"
Jul 24 16:11:13.894559 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.894463    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" containerID="8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6" exitCode=0
Jul 24 16:11:13.894559 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.894518    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"
Jul 24 16:11:13.894684 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.894549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" event={"ID":"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe","Type":"ContainerDied","Data":"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"}
Jul 24 16:11:13.894684 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.894598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9" event={"ID":"d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe","Type":"ContainerDied","Data":"b7b125bc5a7037f88f6e92660b57ac4753f92337fd0ed1b6a7ba40f516cc4d3c"}
Jul 24 16:11:13.895971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.895947    2567 generic.go:356] "Generic (PLEG): container finished" podID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" containerID="6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470" exitCode=0
Jul 24 16:11:13.896096 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.895975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" event={"ID":"46c2c4de-19f5-43f0-976b-d01ef900c9a0","Type":"ContainerDied","Data":"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"}
Jul 24 16:11:13.896096 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.896014    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"
Jul 24 16:11:13.896096 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.896025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j" event={"ID":"46c2c4de-19f5-43f0-976b-d01ef900c9a0","Type":"ContainerDied","Data":"9c05dd3a3d65108b32a83c8da350e0b5e58c96cf0134607a6a07c894706a6bd5"}
Jul 24 16:11:13.902358 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.902213    2567 scope.go:117] "RemoveContainer" containerID="8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"
Jul 24 16:11:13.902497 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:13.902475    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769\": container with ID starting with 8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769 not found: ID does not exist" containerID="8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"
Jul 24 16:11:13.902614 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.902502    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769"} err="failed to get container status \"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769\": rpc error: code = NotFound desc = could not find container \"8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769\": container with ID starting with 8eeb957298a05266d57f1dad10ad76f3e1e5190c2596b70bc4962e1e7117c769 not found: ID does not exist"
Jul 24 16:11:13.902614 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.902518    2567 scope.go:117] "RemoveContainer" containerID="8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"
Jul 24 16:11:13.912301 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.912284    2567 scope.go:117] "RemoveContainer" containerID="8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"
Jul 24 16:11:13.912551 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:13.912533    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6\": container with ID starting with 8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6 not found: ID does not exist" containerID="8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"
Jul 24 16:11:13.912638 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.912557    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6"} err="failed to get container status \"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6\": rpc error: code = NotFound desc = could not find container \"8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6\": container with ID starting with 8959000e0d114633d5c86eaf1f0c3ec72f3a2d182f7a96fc1faa9d8765aa3ac6 not found: ID does not exist"
Jul 24 16:11:13.912638 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.912571    2567 scope.go:117] "RemoveContainer" containerID="6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"
Jul 24 16:11:13.918223 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.918197    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:11:13.920454 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.920438    2567 scope.go:117] "RemoveContainer" containerID="6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"
Jul 24 16:11:13.920707 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:13.920688    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470\": container with ID starting with 6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470 not found: ID does not exist" containerID="6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"
Jul 24 16:11:13.920764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.920719    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470"} err="failed to get container status \"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470\": rpc error: code = NotFound desc = could not find container \"6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470\": container with ID starting with 6f21616adc50bf469959c3ca14701292acbd9376c384b43166d5cf7b14054470 not found: ID does not exist"
Jul 24 16:11:13.923443 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.923416    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-e125f3d6-86cd756d9b-tt9bq"]
Jul 24 16:11:13.933604 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.933583    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:11:13.939377 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.939355    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e125f3d6-86c84b4b9b-7c82j"]
Jul 24 16:11:13.949819 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.949794    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:11:13.953778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:13.953754    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e125f3d6-5c64b5cf94-slsz9"]
Jul 24 16:11:14.949029 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:14.948997    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" path="/var/lib/kubelet/pods/44732f64-6aa3-4ea2-b19e-ad2aa07a762e/volumes"
Jul 24 16:11:14.949438 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:14.949368    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" path="/var/lib/kubelet/pods/46c2c4de-19f5-43f0-976b-d01ef900c9a0/volumes"
Jul 24 16:11:14.949712 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:14.949698    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" path="/var/lib/kubelet/pods/d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe/volumes"
Jul 24 16:11:17.335386 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:17.335349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:18.579302 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:18.579267    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:18.801094 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:18.801063    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:19.076145 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:19.076075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:19.388802 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:19.388721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:19.870923 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:19.870887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:21.707381 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.707349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:21.802817 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.802778    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:11:21.803138 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803112    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" containerName="maas-api"
Jul 24 16:11:21.803207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803144    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" containerName="maas-api"
Jul 24 16:11:21.803207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803153    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" containerName="payload-processing"
Jul 24 16:11:21.803207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803159    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" containerName="payload-processing"
Jul 24 16:11:21.803207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803190    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" containerName="payload-pre-processing"
Jul 24 16:11:21.803207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803199    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" containerName="payload-pre-processing"
Jul 24 16:11:21.803487 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803298    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3b107f0-b0bb-4bff-bd29-d7a9c1f0f7fe" containerName="maas-api"
Jul 24 16:11:21.803487 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803310    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="46c2c4de-19f5-43f0-976b-d01ef900c9a0" containerName="payload-processing"
Jul 24 16:11:21.803487 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.803321    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44732f64-6aa3-4ea2-b19e-ad2aa07a762e" containerName="payload-pre-processing"
Jul 24 16:11:21.807930 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.807911    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:21.810947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.810923    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-da03131f-serving-cert\""
Jul 24 16:11:21.817256 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.817197    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:11:21.818862 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.818835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:21.916618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.916581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:21.938031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.937991    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:21.938249 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.938088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:21.938249 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:21.938181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mdh8\" (UniqueName: \"kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.039172 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.039113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.039338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.039207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.039338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.039251    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mdh8\" (UniqueName: \"kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.039338 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:22.039278    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-serving-cert: secret "maas-api-e2e-derive-da03131f-serving-cert" not found
Jul 24 16:11:22.039488 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:22.039360    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls podName:7f6c37bb-95c4-4609-a95e-762ac9f485cf nodeName:}" failed. No retries permitted until 2026-07-24 16:11:22.539340682 +0000 UTC m=+1824.091580433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls") pod "maas-api-e2e-derive-da03131f-564f775688-89dmn" (UID: "7f6c37bb-95c4-4609-a95e-762ac9f485cf") : secret "maas-api-e2e-derive-da03131f-serving-cert" not found
Jul 24 16:11:22.039942 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.039925    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.051550 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.051522    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mdh8\" (UniqueName: \"kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.159395 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.159353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:22.436885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.436772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:22.527669 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.527638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:22.532298 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.532267    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:22.535640 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.535624    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.538586 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.538564    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-da03131f\""
Jul 24 16:11:22.543525 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.543503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.546726 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.546702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") pod \"maas-api-e2e-derive-da03131f-564f775688-89dmn\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.546950 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.546927    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:22.554233 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.554208    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:22.557549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.557530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.562104 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.562085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-da03131f-dockercfg-6dj7z\""
Jul 24 16:11:22.574770 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.574748    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:22.644369 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.644328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl79w\" (UniqueName: \"kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.644549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.644400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.644549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.644429    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdqc2\" (UniqueName: \"kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.644549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.644454    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.720237 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.720151    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:22.745543 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.745510    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl79w\" (UniqueName: \"kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.745672 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.745556    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.745672 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.745581    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdqc2\" (UniqueName: \"kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.745672 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.745614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.746225 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.746202    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.746316 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.746299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.755134 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.755106    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdqc2\" (UniqueName: \"kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2\") pod \"payload-processing-e2e-derive-da03131f-798cb9df4-qq66b\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") " pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.755259 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.755239    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl79w\" (UniqueName: \"kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w\") pod \"payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.847554 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.847516    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:22.869095 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.869069    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:22.877244 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.877220    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:11:22.882384 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:11:22.882356    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f6c37bb_95c4_4609_a95e_762ac9f485cf.slice/crio-0af48a40c5c3472ca14c2a6af841ffea1a6406cfae2b4ca1874d8f73b157ae66 WatchSource:0}: Error finding container 0af48a40c5c3472ca14c2a6af841ffea1a6406cfae2b4ca1874d8f73b157ae66: Status 404 returned error can't find the container with id 0af48a40c5c3472ca14c2a6af841ffea1a6406cfae2b4ca1874d8f73b157ae66
Jul 24 16:11:22.950956 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:22.950835    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" event={"ID":"7f6c37bb-95c4-4609-a95e-762ac9f485cf","Type":"ContainerStarted","Data":"0af48a40c5c3472ca14c2a6af841ffea1a6406cfae2b4ca1874d8f73b157ae66"}
Jul 24 16:11:23.007257 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.007231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:23.009320 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:11:23.009273    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b90811_5a4a_48f4_b5d2_60dbf5d2de71.slice/crio-27def50535e40412794f5562636064cd6670919d5d920a592090150a8b30c5b5 WatchSource:0}: Error finding container 27def50535e40412794f5562636064cd6670919d5d920a592090150a8b30c5b5: Status 404 returned error can't find the container with id 27def50535e40412794f5562636064cd6670919d5d920a592090150a8b30c5b5
Jul 24 16:11:23.245935 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.245909    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:23.248336 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:11:23.248305    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e439f8_128c_47ba_9c70_907d143aa59f.slice/crio-af877e73a944a4dbac58be7e41b36aa48a290a8fff3fd9be1f36fa3f3be5aaab WatchSource:0}: Error finding container af877e73a944a4dbac58be7e41b36aa48a290a8fff3fd9be1f36fa3f3be5aaab: Status 404 returned error can't find the container with id af877e73a944a4dbac58be7e41b36aa48a290a8fff3fd9be1f36fa3f3be5aaab
Jul 24 16:11:23.954191 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.954072    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" event={"ID":"52b90811-5a4a-48f4-b5d2-60dbf5d2de71","Type":"ContainerStarted","Data":"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"}
Jul 24 16:11:23.954191 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.954169    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" event={"ID":"52b90811-5a4a-48f4-b5d2-60dbf5d2de71","Type":"ContainerStarted","Data":"27def50535e40412794f5562636064cd6670919d5d920a592090150a8b30c5b5"}
Jul 24 16:11:23.954649 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.954202    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:23.955791 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.955764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" event={"ID":"7f6c37bb-95c4-4609-a95e-762ac9f485cf","Type":"ContainerStarted","Data":"1f0fe8d038a6178e7acdf403e09f016d0d24b6b25396d5e8ac26c5aa473ca736"}
Jul 24 16:11:23.955992 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.955852    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:23.957242 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.957218    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" event={"ID":"19e439f8-128c-47ba-9c70-907d143aa59f","Type":"ContainerStarted","Data":"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"}
Jul 24 16:11:23.957358 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.957257    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" event={"ID":"19e439f8-128c-47ba-9c70-907d143aa59f","Type":"ContainerStarted","Data":"af877e73a944a4dbac58be7e41b36aa48a290a8fff3fd9be1f36fa3f3be5aaab"}
Jul 24 16:11:23.957358 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.957287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:23.972473 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.972419    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" podStartSLOduration=1.972399909 podStartE2EDuration="1.972399909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:23.971463597 +0000 UTC m=+1825.523703371" watchObservedRunningTime="2026-07-24 16:11:23.972399909 +0000 UTC m=+1825.524639681"
Jul 24 16:11:23.989481 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:23.989419    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" podStartSLOduration=1.989400587 podStartE2EDuration="1.989400587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:23.987663408 +0000 UTC m=+1825.539903183" watchObservedRunningTime="2026-07-24 16:11:23.989400587 +0000 UTC m=+1825.541640360"
Jul 24 16:11:24.006484 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:24.006428    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" podStartSLOduration=2.419232486 podStartE2EDuration="3.006409262s" totalImagesPullingTime="587.176776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:24.005098989 +0000 UTC m=+1825.557338828" watchObservedRunningTime="2026-07-24 16:11:24.006409262 +0000 UTC m=+1825.558649064"
Jul 24 16:11:29.967167 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:29.967120    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:34.963345 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:34.963314    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:34.963752 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:34.963406    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:57.980839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:57.980803    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:11:57.981388 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:57.981036    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" podUID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" containerName="maas-api" containerID="cri-o://1f0fe8d038a6178e7acdf403e09f016d0d24b6b25396d5e8ac26c5aa473ca736" gracePeriod=30
Jul 24 16:11:58.008038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.007998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:58.042238 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.042185    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:58.042519 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.042492    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" podUID="19e439f8-128c-47ba-9c70-907d143aa59f" containerName="payload-processing" containerID="cri-o://9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e" gracePeriod=30
Jul 24 16:11:58.048110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.048073    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:58.048552 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.048478    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" podUID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" containerName="payload-pre-processing" containerID="cri-o://4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f" gracePeriod=30
Jul 24 16:11:58.083527 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.083488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:58.123727 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.123691    2567 generic.go:356] "Generic (PLEG): container finished" podID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" containerID="1f0fe8d038a6178e7acdf403e09f016d0d24b6b25396d5e8ac26c5aa473ca736" exitCode=0
Jul 24 16:11:58.123918 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.123769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" event={"ID":"7f6c37bb-95c4-4609-a95e-762ac9f485cf","Type":"ContainerDied","Data":"1f0fe8d038a6178e7acdf403e09f016d0d24b6b25396d5e8ac26c5aa473ca736"}
Jul 24 16:11:58.143933 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.143896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:11:58.316409 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.316378    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:58.337584 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.337552    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:58.362646 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.362609    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mdh8\" (UniqueName: \"kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8\") pod \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") "
Jul 24 16:11:58.362887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.362737    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca\") pod \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") "
Jul 24 16:11:58.362887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.362768    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume\") pod \"19e439f8-128c-47ba-9c70-907d143aa59f\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") "
Jul 24 16:11:58.362887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.362810    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") pod \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\" (UID: \"7f6c37bb-95c4-4609-a95e-762ac9f485cf\") "
Jul 24 16:11:58.362887 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.362875    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdqc2\" (UniqueName: \"kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2\") pod \"19e439f8-128c-47ba-9c70-907d143aa59f\" (UID: \"19e439f8-128c-47ba-9c70-907d143aa59f\") "
Jul 24 16:11:58.363233 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.363164    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f6c37bb-95c4-4609-a95e-762ac9f485cf" (UID: "7f6c37bb-95c4-4609-a95e-762ac9f485cf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:58.363302 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.363263    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19e439f8-128c-47ba-9c70-907d143aa59f" (UID: "19e439f8-128c-47ba-9c70-907d143aa59f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:58.365076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.365050    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f6c37bb-95c4-4609-a95e-762ac9f485cf" (UID: "7f6c37bb-95c4-4609-a95e-762ac9f485cf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:11:58.365342 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.365323    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8" (OuterVolumeSpecName: "kube-api-access-5mdh8") pod "7f6c37bb-95c4-4609-a95e-762ac9f485cf" (UID: "7f6c37bb-95c4-4609-a95e-762ac9f485cf"). InnerVolumeSpecName "kube-api-access-5mdh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:58.365502 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.365472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2" (OuterVolumeSpecName: "kube-api-access-zdqc2") pod "19e439f8-128c-47ba-9c70-907d143aa59f" (UID: "19e439f8-128c-47ba-9c70-907d143aa59f"). InnerVolumeSpecName "kube-api-access-zdqc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:58.372247 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.372213    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:58.464218 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cl79w\" (UniqueName: \"kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w\") pod \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") "
Jul 24 16:11:58.464402 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume\") pod \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\" (UID: \"52b90811-5a4a-48f4-b5d2-60dbf5d2de71\") "
Jul 24 16:11:58.464462 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464450    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdqc2\" (UniqueName: \"kubernetes.io/projected/19e439f8-128c-47ba-9c70-907d143aa59f-kube-api-access-zdqc2\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.464500 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464464    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mdh8\" (UniqueName: \"kubernetes.io/projected/7f6c37bb-95c4-4609-a95e-762ac9f485cf-kube-api-access-5mdh8\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.464500 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464473    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f6c37bb-95c4-4609-a95e-762ac9f485cf-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.464500 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464482    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19e439f8-128c-47ba-9c70-907d143aa59f-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.464500 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464491    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f6c37bb-95c4-4609-a95e-762ac9f485cf-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.464711 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.464688    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52b90811-5a4a-48f4-b5d2-60dbf5d2de71" (UID: "52b90811-5a4a-48f4-b5d2-60dbf5d2de71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:11:58.466550 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.466528    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w" (OuterVolumeSpecName: "kube-api-access-cl79w") pod "52b90811-5a4a-48f4-b5d2-60dbf5d2de71" (UID: "52b90811-5a4a-48f4-b5d2-60dbf5d2de71"). InnerVolumeSpecName "kube-api-access-cl79w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:11:58.565068 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.564972    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:58.565068 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:58.565001    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl79w\" (UniqueName: \"kubernetes.io/projected/52b90811-5a4a-48f4-b5d2-60dbf5d2de71-kube-api-access-cl79w\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:11:59.129862 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.129830    2567 generic.go:356] "Generic (PLEG): container finished" podID="19e439f8-128c-47ba-9c70-907d143aa59f" containerID="9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e" exitCode=0
Jul 24 16:11:59.130404 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.129916    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"
Jul 24 16:11:59.130404 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.129921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" event={"ID":"19e439f8-128c-47ba-9c70-907d143aa59f","Type":"ContainerDied","Data":"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"}
Jul 24 16:11:59.130404 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.130106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b" event={"ID":"19e439f8-128c-47ba-9c70-907d143aa59f","Type":"ContainerDied","Data":"af877e73a944a4dbac58be7e41b36aa48a290a8fff3fd9be1f36fa3f3be5aaab"}
Jul 24 16:11:59.130404 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.130174    2567 scope.go:117] "RemoveContainer" containerID="9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"
Jul 24 16:11:59.131789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.131761    2567 generic.go:356] "Generic (PLEG): container finished" podID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" containerID="4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f" exitCode=0
Jul 24 16:11:59.131947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.131803    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" event={"ID":"52b90811-5a4a-48f4-b5d2-60dbf5d2de71","Type":"ContainerDied","Data":"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"}
Jul 24 16:11:59.131947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.131851    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"
Jul 24 16:11:59.131947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.131890    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v" event={"ID":"52b90811-5a4a-48f4-b5d2-60dbf5d2de71","Type":"ContainerDied","Data":"27def50535e40412794f5562636064cd6670919d5d920a592090150a8b30c5b5"}
Jul 24 16:11:59.133739 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.133711    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn" event={"ID":"7f6c37bb-95c4-4609-a95e-762ac9f485cf","Type":"ContainerDied","Data":"0af48a40c5c3472ca14c2a6af841ffea1a6406cfae2b4ca1874d8f73b157ae66"}
Jul 24 16:11:59.133860 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.133749    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"
Jul 24 16:11:59.140447 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.140428    2567 scope.go:117] "RemoveContainer" containerID="9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"
Jul 24 16:11:59.140712 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:59.140694    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e\": container with ID starting with 9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e not found: ID does not exist" containerID="9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"
Jul 24 16:11:59.140752 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.140723    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e"} err="failed to get container status \"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e\": rpc error: code = NotFound desc = could not find container \"9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e\": container with ID starting with 9551a808506296f1378eb82b4f8026a47eb2acc0ea2d6984463596711da9b86e not found: ID does not exist"
Jul 24 16:11:59.140752 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.140741    2567 scope.go:117] "RemoveContainer" containerID="4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"
Jul 24 16:11:59.149095 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.149066    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:59.150727 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.150702    2567 scope.go:117] "RemoveContainer" containerID="4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"
Jul 24 16:11:59.151168 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:11:59.151140    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f\": container with ID starting with 4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f not found: ID does not exist" containerID="4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"
Jul 24 16:11:59.151256 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.151175    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f"} err="failed to get container status \"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f\": rpc error: code = NotFound desc = could not find container \"4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f\": container with ID starting with 4235d06c7a39423918f0c2bcdeb508f4ead5537e2b46fe71190bb0830fce7b4f not found: ID does not exist"
Jul 24 16:11:59.151256 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.151196    2567 scope.go:117] "RemoveContainer" containerID="1f0fe8d038a6178e7acdf403e09f016d0d24b6b25396d5e8ac26c5aa473ca736"
Jul 24 16:11:59.152741 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.152717    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-da03131f-798cb9df4-qq66b"]
Jul 24 16:11:59.164867 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.164841    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:59.168593 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.168567    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-da03131f-58564666f6-cnm2v"]
Jul 24 16:11:59.179513 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.179481    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:11:59.183774 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:11:59.183749    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-da03131f-564f775688-89dmn"]
Jul 24 16:12:00.948929 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:00.948891    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19e439f8-128c-47ba-9c70-907d143aa59f" path="/var/lib/kubelet/pods/19e439f8-128c-47ba-9c70-907d143aa59f/volumes"
Jul 24 16:12:00.949642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:00.949618    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" path="/var/lib/kubelet/pods/52b90811-5a4a-48f4-b5d2-60dbf5d2de71/volumes"
Jul 24 16:12:00.950044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:00.950024    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" path="/var/lib/kubelet/pods/7f6c37bb-95c4-4609-a95e-762ac9f485cf/volumes"
Jul 24 16:12:01.696184 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:01.696115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:01.849598 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:01.849564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:02.089886 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:02.089849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:02.580023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:02.579977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:02.670337 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:02.670305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:02.776719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:02.776686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:04.203944 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:04.203910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:10.019952 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:10.019908    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:10.134800 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:10.134752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:10.223216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:10.223180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:10.322257 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:10.322165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:10.757055 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:10.757024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:36.749735 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:36.749650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:42.836157 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:42.836100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:42.925258 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:42.925225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:43.021563 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:43.021531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:12:58.909935 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:12:58.909901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:04.564026 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:04.563991    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:04.670028 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:04.669989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:04.764564 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:04.764528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:15.290623 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:15.290582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:16.107107 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:16.107074    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:22.407378 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:22.407340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:22.510476 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:22.510442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:22.608554 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:22.608522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:22.710255 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:22.710169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:23.516180 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:23.516121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:23.615986 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:23.615955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:23.718991 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:23.718950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:25.331216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:25.331179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:32.433564 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:32.433532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:32.533401 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:32.533360    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:33.119387 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:33.119353    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:38.991456 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:38.991424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:40.181528 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:40.181492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:43.075484 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.075441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:43.163606 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.163570    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:13:43.164025 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164005    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" containerName="payload-pre-processing"
Jul 24 16:13:43.164097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164030    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" containerName="payload-pre-processing"
Jul 24 16:13:43.164097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164045    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19e439f8-128c-47ba-9c70-907d143aa59f" containerName="payload-processing"
Jul 24 16:13:43.164097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164056    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="19e439f8-128c-47ba-9c70-907d143aa59f" containerName="payload-processing"
Jul 24 16:13:43.164097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164075    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" containerName="maas-api"
Jul 24 16:13:43.164097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164085    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" containerName="maas-api"
Jul 24 16:13:43.164325 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164161    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="52b90811-5a4a-48f4-b5d2-60dbf5d2de71" containerName="payload-pre-processing"
Jul 24 16:13:43.164325 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164177    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f6c37bb-95c4-4609-a95e-762ac9f485cf" containerName="maas-api"
Jul 24 16:13:43.164325 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164185    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="19e439f8-128c-47ba-9c70-907d143aa59f" containerName="payload-processing"
Jul 24 16:13:43.164475 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.164445    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:43.167165 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.167146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.169596 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.169573    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4604a53f-serving-cert\""
Jul 24 16:13:43.180024 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.179995    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:13:43.243573 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.243539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:43.249660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.249615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.249839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.249679    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fwj5\" (UniqueName: \"kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.249839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.249713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.350673 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.350578    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.350673 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.350620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fwj5\" (UniqueName: \"kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.350873 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:13:43.350766    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-serving-cert: secret "maas-api-e2e-mt-4604a53f-serving-cert" not found
Jul 24 16:13:43.350873 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.350814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.350873 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:13:43.350855    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls podName:0ae4d7fc-c002-440b-a78d-924a7fccbe33 nodeName:}" failed. No retries permitted until 2026-07-24 16:13:43.850834179 +0000 UTC m=+1965.403073930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls") pod "maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" (UID: "0ae4d7fc-c002-440b-a78d-924a7fccbe33") : secret "maas-api-e2e-mt-4604a53f-serving-cert" not found
Jul 24 16:13:43.351561 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.351537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.365155 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.365081    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fwj5\" (UniqueName: \"kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.511273 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.511229    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:43.855953 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.855920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.858702 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.858675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") pod \"maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:43.985401 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:43.985359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:44.079018 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.078979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:44.080950 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.080929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:44.183144 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.183100    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:13:44.186802 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.186778    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.189419 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.189395    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4604a53f\""
Jul 24 16:13:44.189567 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.189397    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4604a53f-dockercfg-dclst\""
Jul 24 16:13:44.198284 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.198247    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:13:44.231713 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.231689    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:13:44.234991 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:13:44.234959    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae4d7fc_c002_440b_a78d_924a7fccbe33.slice/crio-03961215b9b90bcfffe81922d5f9d58c1778b77ba120e252ee78c320a21d0215 WatchSource:0}: Error finding container 03961215b9b90bcfffe81922d5f9d58c1778b77ba120e252ee78c320a21d0215: Status 404 returned error can't find the container with id 03961215b9b90bcfffe81922d5f9d58c1778b77ba120e252ee78c320a21d0215
Jul 24 16:13:44.248787 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.248758    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:13:44.252067 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.252047    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.259338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.259309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x55s2\" (UniqueName: \"kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.260449 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.259823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfht5\" (UniqueName: \"kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.260627 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.260609    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.260872 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.260849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.263537 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.263506    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:13:44.361640 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.361597    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x55s2\" (UniqueName: \"kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.361640 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.361645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfht5\" (UniqueName: \"kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.361869 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.361667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.361869 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.361706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.362466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.362446    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.362515 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.362501    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.370558 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.370528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfht5\" (UniqueName: \"kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5\") pod \"payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") " pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.370694 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.370595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x55s2\" (UniqueName: \"kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2\") pod \"payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.500817 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.500776    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:44.565166 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.565119    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:44.621714 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.621605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" event={"ID":"0ae4d7fc-c002-440b-a78d-924a7fccbe33","Type":"ContainerStarted","Data":"03961215b9b90bcfffe81922d5f9d58c1778b77ba120e252ee78c320a21d0215"}
Jul 24 16:13:44.653223 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.653173    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:13:44.655422 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:13:44.655393    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d12aac_dce0_4f84_97a6_53d1787947fe.slice/crio-8fa25e467c3ac92ad41cdd819490deec3ccc822f43f41811f588518055a2c465 WatchSource:0}: Error finding container 8fa25e467c3ac92ad41cdd819490deec3ccc822f43f41811f588518055a2c465: Status 404 returned error can't find the container with id 8fa25e467c3ac92ad41cdd819490deec3ccc822f43f41811f588518055a2c465
Jul 24 16:13:44.725430 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:44.725265    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:13:44.727977 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:13:44.727944    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a55c1d_9c2d_4e41_990c_ebb72722c9fa.slice/crio-68b422c4ac38e2f28e89a6d004e476726b81fb10f321094ce879cdfd9dae145d WatchSource:0}: Error finding container 68b422c4ac38e2f28e89a6d004e476726b81fb10f321094ce879cdfd9dae145d: Status 404 returned error can't find the container with id 68b422c4ac38e2f28e89a6d004e476726b81fb10f321094ce879cdfd9dae145d
Jul 24 16:13:45.628488 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.628443    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" event={"ID":"b8d12aac-dce0-4f84-97a6-53d1787947fe","Type":"ContainerStarted","Data":"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"}
Jul 24 16:13:45.628985 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.628508    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:13:45.628985 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.628529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" event={"ID":"b8d12aac-dce0-4f84-97a6-53d1787947fe","Type":"ContainerStarted","Data":"8fa25e467c3ac92ad41cdd819490deec3ccc822f43f41811f588518055a2c465"}
Jul 24 16:13:45.630150 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.630101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" event={"ID":"0ae4d7fc-c002-440b-a78d-924a7fccbe33","Type":"ContainerStarted","Data":"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"}
Jul 24 16:13:45.630281 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.630191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:45.631622 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.631592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" event={"ID":"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa","Type":"ContainerStarted","Data":"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"}
Jul 24 16:13:45.631759 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.631635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" event={"ID":"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa","Type":"ContainerStarted","Data":"68b422c4ac38e2f28e89a6d004e476726b81fb10f321094ce879cdfd9dae145d"}
Jul 24 16:13:45.631759 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.631747    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:45.645023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.644975    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" podStartSLOduration=1.6449608150000001 podStartE2EDuration="1.644960815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:45.643895078 +0000 UTC m=+1967.196134852" watchObservedRunningTime="2026-07-24 16:13:45.644960815 +0000 UTC m=+1967.197200588"
Jul 24 16:13:45.660410 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.660351    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" podStartSLOduration=2.159340946 podStartE2EDuration="2.660333485s" totalImagesPullingTime="500.992539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:45.659359644 +0000 UTC m=+1967.211599417" watchObservedRunningTime="2026-07-24 16:13:45.660333485 +0000 UTC m=+1967.212573260"
Jul 24 16:13:45.676873 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:45.676819    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" podStartSLOduration=1.676800772 podStartE2EDuration="1.676800772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:45.675665342 +0000 UTC m=+1967.227905117" watchObservedRunningTime="2026-07-24 16:13:45.676800772 +0000 UTC m=+1967.229040545"
Jul 24 16:13:47.564192 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:47.564158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:48.255717 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:48.255679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:48.355515 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:48.355484    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:48.657867 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:48.657731    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:51.643052 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:51.642959    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:13:54.530395 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:54.530359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:54.633289 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:54.633227    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:13:56.638850 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:56.638821    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:13:56.639353 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:13:56.638903    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:14:05.150182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.150148    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:14:05.150628 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.150362    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" podUID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" containerName="maas-api" containerID="cri-o://ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2" gracePeriod=30
Jul 24 16:14:05.173155 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.173089    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:05.195922 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.195880    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:14:05.196253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.196220    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" podUID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" containerName="payload-processing" containerID="cri-o://77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852" gracePeriod=30
Jul 24 16:14:05.201200 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.201161    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:14:05.201473 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.201423    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" podUID="b8d12aac-dce0-4f84-97a6-53d1787947fe" containerName="payload-pre-processing" containerID="cri-o://0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026" gracePeriod=30
Jul 24 16:14:05.272048 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.272007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:05.372868 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.372788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:05.459027 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.458999    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:14:05.484525 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.484336    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:14:05.490307 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.490278    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:14:05.549761 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549730    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfht5\" (UniqueName: \"kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5\") pod \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") "
Jul 24 16:14:05.549957 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549772    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume\") pod \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\" (UID: \"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa\") "
Jul 24 16:14:05.549957 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549794    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fwj5\" (UniqueName: \"kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5\") pod \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") "
Jul 24 16:14:05.549957 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549848    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") pod \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") "
Jul 24 16:14:05.549957 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549871    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume\") pod \"b8d12aac-dce0-4f84-97a6-53d1787947fe\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") "
Jul 24 16:14:05.549957 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.549926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x55s2\" (UniqueName: \"kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2\") pod \"b8d12aac-dce0-4f84-97a6-53d1787947fe\" (UID: \"b8d12aac-dce0-4f84-97a6-53d1787947fe\") "
Jul 24 16:14:05.550274 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550104    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca\") pod \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\" (UID: \"0ae4d7fc-c002-440b-a78d-924a7fccbe33\") "
Jul 24 16:14:05.550274 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550222    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" (UID: "d2a55c1d-9c2d-4e41-990c-ebb72722c9fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:05.550375 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550303    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8d12aac-dce0-4f84-97a6-53d1787947fe" (UID: "b8d12aac-dce0-4f84-97a6-53d1787947fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:05.550549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550490    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d12aac-dce0-4f84-97a6-53d1787947fe-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.550549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550513    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.550549 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.550508    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0ae4d7fc-c002-440b-a78d-924a7fccbe33" (UID: "0ae4d7fc-c002-440b-a78d-924a7fccbe33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:05.552180 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.552158    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0ae4d7fc-c002-440b-a78d-924a7fccbe33" (UID: "0ae4d7fc-c002-440b-a78d-924a7fccbe33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:14:05.552284 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.552181    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5" (OuterVolumeSpecName: "kube-api-access-mfht5") pod "d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" (UID: "d2a55c1d-9c2d-4e41-990c-ebb72722c9fa"). InnerVolumeSpecName "kube-api-access-mfht5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:05.552615 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.552592    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2" (OuterVolumeSpecName: "kube-api-access-x55s2") pod "b8d12aac-dce0-4f84-97a6-53d1787947fe" (UID: "b8d12aac-dce0-4f84-97a6-53d1787947fe"). InnerVolumeSpecName "kube-api-access-x55s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:05.552615 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.552598    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5" (OuterVolumeSpecName: "kube-api-access-7fwj5") pod "0ae4d7fc-c002-440b-a78d-924a7fccbe33" (UID: "0ae4d7fc-c002-440b-a78d-924a7fccbe33"). InnerVolumeSpecName "kube-api-access-7fwj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:05.651105 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.651059    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfht5\" (UniqueName: \"kubernetes.io/projected/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa-kube-api-access-mfht5\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.651105 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.651100    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fwj5\" (UniqueName: \"kubernetes.io/projected/0ae4d7fc-c002-440b-a78d-924a7fccbe33-kube-api-access-7fwj5\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.651105 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.651111    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ae4d7fc-c002-440b-a78d-924a7fccbe33-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.651105 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.651119    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x55s2\" (UniqueName: \"kubernetes.io/projected/b8d12aac-dce0-4f84-97a6-53d1787947fe-kube-api-access-x55s2\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.651424 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.651158    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ae4d7fc-c002-440b-a78d-924a7fccbe33-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:14:05.728817 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.728773    2567 generic.go:356] "Generic (PLEG): container finished" podID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" containerID="ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2" exitCode=0
Jul 24 16:14:05.728996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.728828    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"
Jul 24 16:14:05.728996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.728861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" event={"ID":"0ae4d7fc-c002-440b-a78d-924a7fccbe33","Type":"ContainerDied","Data":"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"}
Jul 24 16:14:05.728996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.728934    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv" event={"ID":"0ae4d7fc-c002-440b-a78d-924a7fccbe33","Type":"ContainerDied","Data":"03961215b9b90bcfffe81922d5f9d58c1778b77ba120e252ee78c320a21d0215"}
Jul 24 16:14:05.728996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.728952    2567 scope.go:117] "RemoveContainer" containerID="ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"
Jul 24 16:14:05.730581 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.730560    2567 generic.go:356] "Generic (PLEG): container finished" podID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" containerID="77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852" exitCode=0
Jul 24 16:14:05.730682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.730617    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"
Jul 24 16:14:05.730682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.730634    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" event={"ID":"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa","Type":"ContainerDied","Data":"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"}
Jul 24 16:14:05.730788 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.730698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw" event={"ID":"d2a55c1d-9c2d-4e41-990c-ebb72722c9fa","Type":"ContainerDied","Data":"68b422c4ac38e2f28e89a6d004e476726b81fb10f321094ce879cdfd9dae145d"}
Jul 24 16:14:05.732310 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.732158    2567 generic.go:356] "Generic (PLEG): container finished" podID="b8d12aac-dce0-4f84-97a6-53d1787947fe" containerID="0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026" exitCode=0
Jul 24 16:14:05.732310 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.732218    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"
Jul 24 16:14:05.732310 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.732248    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" event={"ID":"b8d12aac-dce0-4f84-97a6-53d1787947fe","Type":"ContainerDied","Data":"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"}
Jul 24 16:14:05.732310 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.732299    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8" event={"ID":"b8d12aac-dce0-4f84-97a6-53d1787947fe","Type":"ContainerDied","Data":"8fa25e467c3ac92ad41cdd819490deec3ccc822f43f41811f588518055a2c465"}
Jul 24 16:14:05.740179 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.740157    2567 scope.go:117] "RemoveContainer" containerID="ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"
Jul 24 16:14:05.740524 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:05.740502    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2\": container with ID starting with ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2 not found: ID does not exist" containerID="ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"
Jul 24 16:14:05.740606 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.740533    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2"} err="failed to get container status \"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2\": rpc error: code = NotFound desc = could not find container \"ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2\": container with ID starting with ad5493d4c0fb63b3f7f96ea5e7b7f8450e533453d04a63471beea426583105e2 not found: ID does not exist"
Jul 24 16:14:05.740606 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.740549    2567 scope.go:117] "RemoveContainer" containerID="77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"
Jul 24 16:14:05.750346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.750328    2567 scope.go:117] "RemoveContainer" containerID="77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"
Jul 24 16:14:05.750653 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:05.750634    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852\": container with ID starting with 77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852 not found: ID does not exist" containerID="77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"
Jul 24 16:14:05.750693 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.750662    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852"} err="failed to get container status \"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852\": rpc error: code = NotFound desc = could not find container \"77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852\": container with ID starting with 77306ad4c1ae3cf3ce22a028ca963339a80d6772f2e6df6471880a4a92468852 not found: ID does not exist"
Jul 24 16:14:05.750693 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.750681    2567 scope.go:117] "RemoveContainer" containerID="0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"
Jul 24 16:14:05.759318 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.759289    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:14:05.759959 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.759937    2567 scope.go:117] "RemoveContainer" containerID="0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"
Jul 24 16:14:05.760309 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:05.760290    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026\": container with ID starting with 0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026 not found: ID does not exist" containerID="0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"
Jul 24 16:14:05.760372 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.760318    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026"} err="failed to get container status \"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026\": rpc error: code = NotFound desc = could not find container \"0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026\": container with ID starting with 0269914f1ed509f960742ed944dbc3b572e1bd8f6a681b69dd7de9dd5fc87026 not found: ID does not exist"
Jul 24 16:14:05.761076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.761057    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4604a53f-7dbcd7f46-lsskv"]
Jul 24 16:14:05.770747 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.770712    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:14:05.774388 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.774362    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4604a53f-69f8fb86f8-zp5bw"]
Jul 24 16:14:05.784914 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.784879    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:14:05.788301 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:05.788272    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-4604a53f-674b9c57bd-d5fr8"]
Jul 24 16:14:06.954350 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:06.954313    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" path="/var/lib/kubelet/pods/0ae4d7fc-c002-440b-a78d-924a7fccbe33/volumes"
Jul 24 16:14:06.954746 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:06.954658    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d12aac-dce0-4f84-97a6-53d1787947fe" path="/var/lib/kubelet/pods/b8d12aac-dce0-4f84-97a6-53d1787947fe/volumes"
Jul 24 16:14:06.954954 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:06.954941    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" path="/var/lib/kubelet/pods/d2a55c1d-9c2d-4e41-990c-ebb72722c9fa/volumes"
Jul 24 16:14:09.977552 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:09.977518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:10.365806 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:10.365753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.101752 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.101718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.203225 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.203188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.367427 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.367348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.608789 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.608750    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.745667 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.745595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.836460 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.836428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:16.958373 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:16.958337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:22.890096 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:22.890059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:22.950893 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:22.950857    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:23.040630 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:23.040590    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:23.132991 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:23.132956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:23.263866 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:23.263835    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:28.854511 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:28.854457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:28.953997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:28.953967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:30.851110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:30.851080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:30.949911 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:30.949878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:35.732082 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:35.732037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:36.037587 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:36.037477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:36.191098 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:36.191062    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:38.627943 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:38.627883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:39.352044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.352008    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:39.436077 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.436046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:39.439928 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.439896    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:14:39.440272 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440256    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" containerName="maas-api"
Jul 24 16:14:39.440346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440274    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" containerName="maas-api"
Jul 24 16:14:39.440346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440284    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" containerName="payload-processing"
Jul 24 16:14:39.440346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440291    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" containerName="payload-processing"
Jul 24 16:14:39.440346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440300    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d12aac-dce0-4f84-97a6-53d1787947fe" containerName="payload-pre-processing"
Jul 24 16:14:39.440346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440306    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d12aac-dce0-4f84-97a6-53d1787947fe" containerName="payload-pre-processing"
Jul 24 16:14:39.440607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440362    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d12aac-dce0-4f84-97a6-53d1787947fe" containerName="payload-pre-processing"
Jul 24 16:14:39.440607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440370    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae4d7fc-c002-440b-a78d-924a7fccbe33" containerName="maas-api"
Jul 24 16:14:39.440607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.440375    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a55c1d-9c2d-4e41-990c-ebb72722c9fa" containerName="payload-processing"
Jul 24 16:14:39.443374 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.443354    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.445882 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.445855    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b89f991b-serving-cert\""
Jul 24 16:14:39.455826 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.455791    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:14:39.505246 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.505212    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:39.534614 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.534568    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.534614 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.534620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.534859 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.534640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7znn4\" (UniqueName: \"kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.635861 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.635753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.635861 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.635799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.635861 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.635817    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7znn4\" (UniqueName: \"kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.636437 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:39.635913    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-serving-cert: secret "maas-api-e2e-mt-b89f991b-serving-cert" not found
Jul 24 16:14:39.636437 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:39.635992    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls podName:ca1d00ff-8807-401f-bf5b-b5a929baf511 nodeName:}" failed. No retries permitted until 2026-07-24 16:14:40.13597021 +0000 UTC m=+2021.688209962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls") pod "maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" (UID: "ca1d00ff-8807-401f-bf5b-b5a929baf511") : secret "maas-api-e2e-mt-b89f991b-serving-cert" not found
Jul 24 16:14:39.636437 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.636418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:39.648471 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:39.648438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7znn4\" (UniqueName: \"kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:40.059114 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.059081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:40.139608 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.139573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:40.139802 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:40.139727    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-serving-cert: secret "maas-api-e2e-mt-b89f991b-serving-cert" not found
Jul 24 16:14:40.139848 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:14:40.139802    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls podName:ca1d00ff-8807-401f-bf5b-b5a929baf511 nodeName:}" failed. No retries permitted until 2026-07-24 16:14:41.139783447 +0000 UTC m=+2022.692023199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls") pod "maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" (UID: "ca1d00ff-8807-401f-bf5b-b5a929baf511") : secret "maas-api-e2e-mt-b89f991b-serving-cert" not found
Jul 24 16:14:40.155816 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.155777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:40.240610 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.240568    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:14:40.244271 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.244254    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.249009 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.248985    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b89f991b-dockercfg-7pz54\""
Jul 24 16:14:40.249648 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.249629    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b89f991b\""
Jul 24 16:14:40.257806 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.257776    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:14:40.285076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.285044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:40.342066 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.341976    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.342066 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.342034    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6gfz\" (UniqueName: \"kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.442972 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.442933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.442972 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.442989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6gfz\" (UniqueName: \"kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.443711 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.443686    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.451944 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.451907    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6gfz\" (UniqueName: \"kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz\") pod \"payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.554974 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.554938    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.699347 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.699307    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:14:40.701793 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:14:40.701763    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565417dd_577e_426b_91ba_6b7b0efced68.slice/crio-e80a75aea28d25a4a517cdb0a6c90c2d8ce45722472bc6c6dd52616875a20ab4 WatchSource:0}: Error finding container e80a75aea28d25a4a517cdb0a6c90c2d8ce45722472bc6c6dd52616875a20ab4: Status 404 returned error can't find the container with id e80a75aea28d25a4a517cdb0a6c90c2d8ce45722472bc6c6dd52616875a20ab4
Jul 24 16:14:40.901195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.901051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" event={"ID":"565417dd-577e-426b-91ba-6b7b0efced68","Type":"ContainerStarted","Data":"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"}
Jul 24 16:14:40.901195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.901147    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" event={"ID":"565417dd-577e-426b-91ba-6b7b0efced68","Type":"ContainerStarted","Data":"e80a75aea28d25a4a517cdb0a6c90c2d8ce45722472bc6c6dd52616875a20ab4"}
Jul 24 16:14:40.901195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.901183    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:40.927875 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:40.927810    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" podStartSLOduration=0.927791665 podStartE2EDuration="927.791665ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:40.927328754 +0000 UTC m=+2022.479568526" watchObservedRunningTime="2026-07-24 16:14:40.927791665 +0000 UTC m=+2022.480031440"
Jul 24 16:14:41.149778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.149738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:41.152516 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.152449    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:41.255862 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.255819    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:41.397686 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.397654    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:14:41.400337 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:14:41.400294    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1d00ff_8807_401f_bf5b_b5a929baf511.slice/crio-c459173422997a2241f30892dac88f9bc5a3b54d44bbe6fe164615a96b0d8602 WatchSource:0}: Error finding container c459173422997a2241f30892dac88f9bc5a3b54d44bbe6fe164615a96b0d8602: Status 404 returned error can't find the container with id c459173422997a2241f30892dac88f9bc5a3b54d44bbe6fe164615a96b0d8602
Jul 24 16:14:41.401872 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.401851    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:14:41.908386 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:41.908356    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" event={"ID":"ca1d00ff-8807-401f-bf5b-b5a929baf511","Type":"ContainerStarted","Data":"c459173422997a2241f30892dac88f9bc5a3b54d44bbe6fe164615a96b0d8602"}
Jul 24 16:14:42.915333 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:42.915295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" event={"ID":"ca1d00ff-8807-401f-bf5b-b5a929baf511","Type":"ContainerStarted","Data":"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"}
Jul 24 16:14:42.915762 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:42.915410    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:42.932523 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:42.932467    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" podStartSLOduration=3.483190199 podStartE2EDuration="3.932448868s" totalImagesPullingTime="449.258669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:42.93119971 +0000 UTC m=+2024.483439484" watchObservedRunningTime="2026-07-24 16:14:42.932448868 +0000 UTC m=+2024.484688642"
Jul 24 16:14:43.862605 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:43.862573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:14:48.925420 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:48.925387    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:14:51.910910 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:51.910877    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:14:52.298908 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:14:52.298877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:00.130976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.130929    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"]
Jul 24 16:15:00.136041 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.136011    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"]
Jul 24 16:15:00.136243 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.136144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"
Jul 24 16:15:00.138661 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.138639    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:15:00.139364 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.139345    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"
Jul 24 16:15:00.142649 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.142623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"]
Jul 24 16:15:00.146813 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.146766    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"]
Jul 24 16:15:00.150302 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.150281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"
Jul 24 16:15:00.158893 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.158870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"
Jul 24 16:15:00.328663 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.328636    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"]
Jul 24 16:15:00.331462 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:15:00.331426    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986c9d18_b1a3_4dce_a216_3023aacc4125.slice/crio-89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe WatchSource:0}: Error finding container 89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe: Status 404 returned error can't find the container with id 89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe
Jul 24 16:15:00.344986 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.344963    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"]
Jul 24 16:15:00.346239 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:15:00.346212    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10907ab1_8bde_4795_80fb_419859397f5a.slice/crio-bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9 WatchSource:0}: Error finding container bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9: Status 404 returned error can't find the container with id bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9
Jul 24 16:15:00.999120 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:00.999085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh" event={"ID":"10907ab1-8bde-4795-80fb-419859397f5a","Type":"ContainerStarted","Data":"bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9"}
Jul 24 16:15:01.000243 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:01.000219    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj" event={"ID":"986c9d18-b1a3-4dce-a216-3023aacc4125","Type":"ContainerStarted","Data":"89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe"}
Jul 24 16:15:04.021635 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:04.021600    2567 generic.go:356] "Generic (PLEG): container finished" podID="986c9d18-b1a3-4dce-a216-3023aacc4125" containerID="2924afb3bd1c864052b187fb2578b9417c2895332c72caaf206c75ffb5aa0f6d" exitCode=0
Jul 24 16:15:04.022167 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:04.021692    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj" event={"ID":"986c9d18-b1a3-4dce-a216-3023aacc4125","Type":"ContainerDied","Data":"2924afb3bd1c864052b187fb2578b9417c2895332c72caaf206c75ffb5aa0f6d"}
Jul 24 16:15:04.023391 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:04.023366    2567 generic.go:356] "Generic (PLEG): container finished" podID="10907ab1-8bde-4795-80fb-419859397f5a" containerID="f87c10d87ff0b4f2aa2dbe77002da9e7e536ac673cddf989b8be8d04511b60b2" exitCode=0
Jul 24 16:15:04.023498 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:04.023415    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh" event={"ID":"10907ab1-8bde-4795-80fb-419859397f5a","Type":"ContainerDied","Data":"f87c10d87ff0b4f2aa2dbe77002da9e7e536ac673cddf989b8be8d04511b60b2"}
Jul 24 16:15:05.173927 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:05.173899    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"
Jul 24 16:15:05.196936 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:05.196911    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"
Jul 24 16:15:06.034869 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.034836    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh"
Jul 24 16:15:06.035078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.034843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748495-7xfsh" event={"ID":"10907ab1-8bde-4795-80fb-419859397f5a","Type":"ContainerDied","Data":"bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9"}
Jul 24 16:15:06.035078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.034963    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd48e9fca059a6b319409bb2befc4736d3b6ab3b3d5664b255520c14457764c9"
Jul 24 16:15:06.036142 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.036106    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"
Jul 24 16:15:06.036253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.036147    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj" event={"ID":"986c9d18-b1a3-4dce-a216-3023aacc4125","Type":"ContainerDied","Data":"89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe"}
Jul 24 16:15:06.036253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:06.036180    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89b9df2e4a1548030955f943b5f6bd46d9f50e0d454dda6628162c390ca9e6fe"
Jul 24 16:15:28.623658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.623571    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"]
Jul 24 16:15:28.624031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.623934    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="986c9d18-b1a3-4dce-a216-3023aacc4125" containerName="cleanup"
Jul 24 16:15:28.624031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.623947    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="986c9d18-b1a3-4dce-a216-3023aacc4125" containerName="cleanup"
Jul 24 16:15:28.624031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.623985    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10907ab1-8bde-4795-80fb-419859397f5a" containerName="cleanup"
Jul 24 16:15:28.624031 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.623991    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="10907ab1-8bde-4795-80fb-419859397f5a" containerName="cleanup"
Jul 24 16:15:28.624218 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.624042    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="10907ab1-8bde-4795-80fb-419859397f5a" containerName="cleanup"
Jul 24 16:15:28.624218 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.624053    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="986c9d18-b1a3-4dce-a216-3023aacc4125" containerName="cleanup"
Jul 24 16:15:28.632407 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.632385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:28.635270 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.635237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"]
Jul 24 16:15:28.635615 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.635597    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:15:28.681985 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.681956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b89f991b-m42vs\" (UID: \"d91ea725-b0dd-4423-9315-3a2393b2d1ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:28.783209 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.783162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b89f991b-m42vs\" (UID: \"d91ea725-b0dd-4423-9315-3a2393b2d1ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:28.783764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.783745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b89f991b-m42vs\" (UID: \"d91ea725-b0dd-4423-9315-3a2393b2d1ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:28.945447 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:28.945361    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:29.083114 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:29.083082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"]
Jul 24 16:15:29.085245 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:15:29.085219    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91ea725_b0dd_4423_9315_3a2393b2d1ad.slice/crio-4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498 WatchSource:0}: Error finding container 4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498: Status 404 returned error can't find the container with id 4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498
Jul 24 16:15:29.154488 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:29.154451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs" event={"ID":"d91ea725-b0dd-4423-9315-3a2393b2d1ad","Type":"ContainerStarted","Data":"4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498"}
Jul 24 16:15:30.160441 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:30.160357    2567 generic.go:356] "Generic (PLEG): container finished" podID="d91ea725-b0dd-4423-9315-3a2393b2d1ad" containerID="4a91c608c3215d1b7f415d50e1ae4d298c646355814a9c87ee5ad1682ff882bd" exitCode=0
Jul 24 16:15:30.160870 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:30.160444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs" event={"ID":"d91ea725-b0dd-4423-9315-3a2393b2d1ad","Type":"ContainerDied","Data":"4a91c608c3215d1b7f415d50e1ae4d298c646355814a9c87ee5ad1682ff882bd"}
Jul 24 16:15:31.301060 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:31.301032    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:31.409229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:31.409188    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca\") pod \"d91ea725-b0dd-4423-9315-3a2393b2d1ad\" (UID: \"d91ea725-b0dd-4423-9315-3a2393b2d1ad\") "
Jul 24 16:15:31.409540 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:31.409515    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d91ea725-b0dd-4423-9315-3a2393b2d1ad" (UID: "d91ea725-b0dd-4423-9315-3a2393b2d1ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:31.510187 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:31.510155    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d91ea725-b0dd-4423-9315-3a2393b2d1ad-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:32.172325 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:32.172298    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"
Jul 24 16:15:32.172492 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:32.172340    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs" event={"ID":"d91ea725-b0dd-4423-9315-3a2393b2d1ad","Type":"ContainerDied","Data":"4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498"}
Jul 24 16:15:32.172492 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:32.172385    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4feb6c6b142fb0edc53289ddf5fcfd04b7b560994540aa778993ce900fc53498"
Jul 24 16:15:44.226673 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.226635    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:15:44.227850 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.227763    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" podUID="ca1d00ff-8807-401f-bf5b-b5a929baf511" containerName="maas-api" containerID="cri-o://78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4" gracePeriod=30
Jul 24 16:15:44.255996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.255962    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"]
Jul 24 16:15:44.259665 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.259639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:44.264238 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.264208    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-b89f991b-29748495-nh9fj"]
Jul 24 16:15:44.318831 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.318795    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:15:44.319082 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.319025    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" podUID="565417dd-577e-426b-91ba-6b7b0efced68" containerName="payload-pre-processing" containerID="cri-o://a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71" gracePeriod=30
Jul 24 16:15:44.358295 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.358252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:44.459505 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.459467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:44.580454 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.580427    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:15:44.601420 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.601393    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:15:44.724755 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.724717    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7znn4\" (UniqueName: \"kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4\") pod \"ca1d00ff-8807-401f-bf5b-b5a929baf511\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") "
Jul 24 16:15:44.724755 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.724758    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume\") pod \"565417dd-577e-426b-91ba-6b7b0efced68\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") "
Jul 24 16:15:44.724958 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.724776    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6gfz\" (UniqueName: \"kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz\") pod \"565417dd-577e-426b-91ba-6b7b0efced68\" (UID: \"565417dd-577e-426b-91ba-6b7b0efced68\") "
Jul 24 16:15:44.724958 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.724840    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca\") pod \"ca1d00ff-8807-401f-bf5b-b5a929baf511\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") "
Jul 24 16:15:44.724958 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.724905    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") pod \"ca1d00ff-8807-401f-bf5b-b5a929baf511\" (UID: \"ca1d00ff-8807-401f-bf5b-b5a929baf511\") "
Jul 24 16:15:44.725261 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.725223    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "565417dd-577e-426b-91ba-6b7b0efced68" (UID: "565417dd-577e-426b-91ba-6b7b0efced68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:44.725389 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.725233    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca1d00ff-8807-401f-bf5b-b5a929baf511" (UID: "ca1d00ff-8807-401f-bf5b-b5a929baf511"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:44.727180 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.727156    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4" (OuterVolumeSpecName: "kube-api-access-7znn4") pod "ca1d00ff-8807-401f-bf5b-b5a929baf511" (UID: "ca1d00ff-8807-401f-bf5b-b5a929baf511"). InnerVolumeSpecName "kube-api-access-7znn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:44.727272 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.727199    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz" (OuterVolumeSpecName: "kube-api-access-c6gfz") pod "565417dd-577e-426b-91ba-6b7b0efced68" (UID: "565417dd-577e-426b-91ba-6b7b0efced68"). InnerVolumeSpecName "kube-api-access-c6gfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:44.727272 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.727218    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca1d00ff-8807-401f-bf5b-b5a929baf511" (UID: "ca1d00ff-8807-401f-bf5b-b5a929baf511"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:15:44.826055 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.826021    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca1d00ff-8807-401f-bf5b-b5a929baf511-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:44.826055 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.826054    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7znn4\" (UniqueName: \"kubernetes.io/projected/ca1d00ff-8807-401f-bf5b-b5a929baf511-kube-api-access-7znn4\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:44.826055 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.826064    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/565417dd-577e-426b-91ba-6b7b0efced68-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:44.826328 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.826073    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6gfz\" (UniqueName: \"kubernetes.io/projected/565417dd-577e-426b-91ba-6b7b0efced68-kube-api-access-c6gfz\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:44.826328 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.826081    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca1d00ff-8807-401f-bf5b-b5a929baf511-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:15:44.949100 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:44.949067    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="986c9d18-b1a3-4dce-a216-3023aacc4125" path="/var/lib/kubelet/pods/986c9d18-b1a3-4dce-a216-3023aacc4125/volumes"
Jul 24 16:15:45.243028 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.242997    2567 generic.go:356] "Generic (PLEG): container finished" podID="565417dd-577e-426b-91ba-6b7b0efced68" containerID="a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71" exitCode=0
Jul 24 16:15:45.243516 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.243067    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"
Jul 24 16:15:45.243516 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.243083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" event={"ID":"565417dd-577e-426b-91ba-6b7b0efced68","Type":"ContainerDied","Data":"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"}
Jul 24 16:15:45.243516 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.243175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p" event={"ID":"565417dd-577e-426b-91ba-6b7b0efced68","Type":"ContainerDied","Data":"e80a75aea28d25a4a517cdb0a6c90c2d8ce45722472bc6c6dd52616875a20ab4"}
Jul 24 16:15:45.243516 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.243209    2567 scope.go:117] "RemoveContainer" containerID="a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"
Jul 24 16:15:45.244658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.244455    2567 generic.go:356] "Generic (PLEG): container finished" podID="ca1d00ff-8807-401f-bf5b-b5a929baf511" containerID="78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4" exitCode=0
Jul 24 16:15:45.244658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.244494    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"
Jul 24 16:15:45.244658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.244555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" event={"ID":"ca1d00ff-8807-401f-bf5b-b5a929baf511","Type":"ContainerDied","Data":"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"}
Jul 24 16:15:45.244658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.244637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5" event={"ID":"ca1d00ff-8807-401f-bf5b-b5a929baf511","Type":"ContainerDied","Data":"c459173422997a2241f30892dac88f9bc5a3b54d44bbe6fe164615a96b0d8602"}
Jul 24 16:15:45.253701 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.253684    2567 scope.go:117] "RemoveContainer" containerID="a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"
Jul 24 16:15:45.254014 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:15:45.253994    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71\": container with ID starting with a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71 not found: ID does not exist" containerID="a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"
Jul 24 16:15:45.254062 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.254024    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71"} err="failed to get container status \"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71\": rpc error: code = NotFound desc = could not find container \"a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71\": container with ID starting with a36b0e9fa8649b5de6f87a397830d8f68e866d9b13a4cbc55d907604c004ac71 not found: ID does not exist"
Jul 24 16:15:45.254062 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.254042    2567 scope.go:117] "RemoveContainer" containerID="78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"
Jul 24 16:15:45.262173 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.262148    2567 scope.go:117] "RemoveContainer" containerID="78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"
Jul 24 16:15:45.262542 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:15:45.262511    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4\": container with ID starting with 78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4 not found: ID does not exist" containerID="78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"
Jul 24 16:15:45.262633 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.262555    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4"} err="failed to get container status \"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4\": rpc error: code = NotFound desc = could not find container \"78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4\": container with ID starting with 78f58a1d262c3b02abb5bfd58d2a45565507ee9cd2b4d129e54082e7ecd765b4 not found: ID does not exist"
Jul 24 16:15:45.264291 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.264267    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:15:45.270548 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.270523    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b89f991b-8c49b4f5-nmkg5"]
Jul 24 16:15:45.280442 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.280416    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:15:45.284342 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:45.284317    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b89f991b-665cd48cb-wdh2p"]
Jul 24 16:15:46.949373 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:46.949343    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="565417dd-577e-426b-91ba-6b7b0efced68" path="/var/lib/kubelet/pods/565417dd-577e-426b-91ba-6b7b0efced68/volumes"
Jul 24 16:15:46.949847 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:46.949716    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca1d00ff-8807-401f-bf5b-b5a929baf511" path="/var/lib/kubelet/pods/ca1d00ff-8807-401f-bf5b-b5a929baf511/volumes"
Jul 24 16:15:48.713115 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:48.713068    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"]
Jul 24 16:15:48.713747 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:48.713719    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:48.718797 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:48.718770    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b89f991b-m42vs"]
Jul 24 16:15:48.949073 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:48.949045    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d91ea725-b0dd-4423-9315-3a2393b2d1ad" path="/var/lib/kubelet/pods/d91ea725-b0dd-4423-9315-3a2393b2d1ad/volumes"
Jul 24 16:15:51.980565 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:51.980525    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:52.087732 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:52.087687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:57.051295 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:57.051235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:15:59.073575 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:59.073530    2567 scope.go:117] "RemoveContainer" containerID="4af519f43555ee0a739c83ac0f8a6f558502ca689ee5b7cb07bbec92fa9909c6"
Jul 24 16:15:59.081946 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:15:59.081924    2567 scope.go:117] "RemoveContainer" containerID="6dfeb981d1d3edefc13555f0ea77969fe52f6fe09c766ab87497e303c47d31b6"
Jul 24 16:16:09.959186 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:09.959105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:10.059966 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:10.059929    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:33.246606 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:33.246567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:34.007528 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:34.007490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:39.501790 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:39.501742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:39.601744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:39.601705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:39.796849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:39.796736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:40.060041 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:40.059933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:41.122335 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:41.122305    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:41.242199 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:41.242150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:41.344200 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:41.344166    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:42.068274 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:42.068236    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:42.157146 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:42.157097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:42.255104 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:42.255068    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:43.602870 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:43.602831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:49.730120 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:49.730020    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:49.789225 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:49.789186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:50.393679 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:50.393643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:56.073839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:56.073802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:56.174712 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:56.174663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:56.305435 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:56.305398    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:16:56.490205 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:16:56.490172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:06.824611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:06.824570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:07.148481 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:07.148392    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:28.129493 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:28.129447    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:34.647727 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:34.647691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:34.741166 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:34.741100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:34.992624 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:34.992589    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:35.336429 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:35.336347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:35.574564 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:35.574528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:38.374466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.374399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:38.455919 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.455881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:38.461185 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461156    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:17:38.461583 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461564    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1d00ff-8807-401f-bf5b-b5a929baf511" containerName="maas-api"
Jul 24 16:17:38.461671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461587    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1d00ff-8807-401f-bf5b-b5a929baf511" containerName="maas-api"
Jul 24 16:17:38.461671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461607    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d91ea725-b0dd-4423-9315-3a2393b2d1ad" containerName="revoke-keys"
Jul 24 16:17:38.461671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461616    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d91ea725-b0dd-4423-9315-3a2393b2d1ad" containerName="revoke-keys"
Jul 24 16:17:38.461671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461634    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="565417dd-577e-426b-91ba-6b7b0efced68" containerName="payload-pre-processing"
Jul 24 16:17:38.461671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461643    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="565417dd-577e-426b-91ba-6b7b0efced68" containerName="payload-pre-processing"
Jul 24 16:17:38.461923 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461736    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="565417dd-577e-426b-91ba-6b7b0efced68" containerName="payload-pre-processing"
Jul 24 16:17:38.461923 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461750    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d91ea725-b0dd-4423-9315-3a2393b2d1ad" containerName="revoke-keys"
Jul 24 16:17:38.461923 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.461764    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca1d00ff-8807-401f-bf5b-b5a929baf511" containerName="maas-api"
Jul 24 16:17:38.464778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.464755    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.467445 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.467423    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-35cc2d-serving-cert\""
Jul 24 16:17:38.477353 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.477327    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:17:38.528037 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.528005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:38.620953 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.620913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.621206 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.620966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.621206 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.621037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbcgf\" (UniqueName: \"kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.624971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.624900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:38.721895 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.721853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.721895 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.721896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbcgf\" (UniqueName: \"kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.722171 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.721962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.722642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.722618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.724688 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.724668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.730540 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.730515    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbcgf\" (UniqueName: \"kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf\") pod \"maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.775847 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.775817    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:38.918413 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:38.918383    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:17:38.921079 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:17:38.921049    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec617639_163d_4029_9b6f_6b2ef8e0ae6d.slice/crio-0a11cefa1503a452a2e7560bb1ae873bbbb7d9b67a5574a8a1faf27902c67581 WatchSource:0}: Error finding container 0a11cefa1503a452a2e7560bb1ae873bbbb7d9b67a5574a8a1faf27902c67581: Status 404 returned error can't find the container with id 0a11cefa1503a452a2e7560bb1ae873bbbb7d9b67a5574a8a1faf27902c67581
Jul 24 16:17:39.155997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.155955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:39.263182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.263114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:39.276001 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.275960    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:17:39.280524 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.280501    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.285902 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.285875    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-35cc2d-dockercfg-v4dr9\""
Jul 24 16:17:39.285902 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.285896    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-35cc2d\""
Jul 24 16:17:39.296504 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.296467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:17:39.429083 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.429041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.429564 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.429094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24n2\" (UniqueName: \"kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.530168 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.530046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.530168 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.530099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24n2\" (UniqueName: \"kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.530800 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.530779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.538717 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.538691    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24n2\" (UniqueName: \"kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2\") pod \"payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.591601 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.591559    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:39.770394 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:17:39.770356    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda11fdf6d_b0c4_4a4d_b442_d78fdcef9758.slice/crio-677d8467cec1b836f76af828f89e0d56c663fad9471fce797e1963a1a90b0f52 WatchSource:0}: Error finding container 677d8467cec1b836f76af828f89e0d56c663fad9471fce797e1963a1a90b0f52: Status 404 returned error can't find the container with id 677d8467cec1b836f76af828f89e0d56c663fad9471fce797e1963a1a90b0f52
Jul 24 16:17:39.771754 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.771727    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:17:39.781613 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.781529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" event={"ID":"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758","Type":"ContainerStarted","Data":"677d8467cec1b836f76af828f89e0d56c663fad9471fce797e1963a1a90b0f52"}
Jul 24 16:17:39.783015 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.782995    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" event={"ID":"ec617639-163d-4029-9b6f-6b2ef8e0ae6d","Type":"ContainerStarted","Data":"228f21158fa56921edc243a67d29e35c6e1d45b369dc9ff6524b5e8531a632f5"}
Jul 24 16:17:39.783152 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.783038    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" event={"ID":"ec617639-163d-4029-9b6f-6b2ef8e0ae6d","Type":"ContainerStarted","Data":"0a11cefa1503a452a2e7560bb1ae873bbbb7d9b67a5574a8a1faf27902c67581"}
Jul 24 16:17:39.783152 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.783111    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:39.820524 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:39.820467    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" podStartSLOduration=1.435248277 podStartE2EDuration="1.820443592s" totalImagesPullingTime="385.195315ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:39.820073354 +0000 UTC m=+2201.372313126" watchObservedRunningTime="2026-07-24 16:17:39.820443592 +0000 UTC m=+2201.372683367"
Jul 24 16:17:40.791909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:40.791867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" event={"ID":"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758","Type":"ContainerStarted","Data":"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"}
Jul 24 16:17:40.792439 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:40.791939    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:40.808255 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:40.808195    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" podStartSLOduration=1.808173534 podStartE2EDuration="1.808173534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:40.807605854 +0000 UTC m=+2202.359845621" watchObservedRunningTime="2026-07-24 16:17:40.808173534 +0000 UTC m=+2202.360413309"
Jul 24 16:17:42.908381 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:42.908341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:43.235263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:43.235224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:43.529893 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:43.529792    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:43.662735 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:43.662684    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:44.470658 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:44.470622    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:45.797864 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:45.797832    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:17:46.425744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.425710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:46.504554 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.504521    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:46.642971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.642934    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:17:46.646721 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.646694    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.649355 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.649326    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-47a0d8-serving-cert\""
Jul 24 16:17:46.658765 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.658434    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:17:46.678991 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.678889    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:46.796810 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.796768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.796810 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.796808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dkfz\" (UniqueName: \"kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.797043 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.796831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.864561 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.864509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:46.897929 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.897879    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.897929 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.897927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dkfz\" (UniqueName: \"kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.898236 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.897952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.898569 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.898545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.900770 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.900743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.909303 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.909267    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dkfz\" (UniqueName: \"kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz\") pod \"maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:46.958492 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:46.958391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:47.121708 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.121670    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:17:47.124265 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:17:47.124219    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57cae15e_ede9_4f68_810a_d0d5e704ca27.slice/crio-56046cb9130af619f101bb8f2980217633f6313a374b3860666110539a2f4cbc WatchSource:0}: Error finding container 56046cb9130af619f101bb8f2980217633f6313a374b3860666110539a2f4cbc: Status 404 returned error can't find the container with id 56046cb9130af619f101bb8f2980217633f6313a374b3860666110539a2f4cbc
Jul 24 16:17:47.282247 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.282208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:47.382931 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.382895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:47.404776 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.404731    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:17:47.409487 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.409457    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.412334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.412308    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-47a0d8\""
Jul 24 16:17:47.412512 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.412496    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-47a0d8-dockercfg-pp8b7\""
Jul 24 16:17:47.418840 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.418520    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:17:47.502630 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.502589    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.502819 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.502657    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz44c\" (UniqueName: \"kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.603678 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.603635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.603881 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.603711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz44c\" (UniqueName: \"kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.604551 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.604524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.612818 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.612786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz44c\" (UniqueName: \"kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c\") pod \"payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.723584 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.723530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:47.828810 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.828761    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" event={"ID":"57cae15e-ede9-4f68-810a-d0d5e704ca27","Type":"ContainerStarted","Data":"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"}
Jul 24 16:17:47.828989 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.828820    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" event={"ID":"57cae15e-ede9-4f68-810a-d0d5e704ca27","Type":"ContainerStarted","Data":"56046cb9130af619f101bb8f2980217633f6313a374b3860666110539a2f4cbc"}
Jul 24 16:17:47.828989 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.828878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:47.849907 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.849835    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" podStartSLOduration=1.461387441 podStartE2EDuration="1.849809649s" totalImagesPullingTime="388.422208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:47.848664339 +0000 UTC m=+2209.400904114" watchObservedRunningTime="2026-07-24 16:17:47.849809649 +0000 UTC m=+2209.402049435"
Jul 24 16:17:47.877682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:47.877649    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:17:47.880830 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:17:47.880799    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4e2241_174c_478d_84a2_958b643730bf.slice/crio-2f1294dfb882381793f726381e9084df1d02e792fce6e03f27537848b67cce96 WatchSource:0}: Error finding container 2f1294dfb882381793f726381e9084df1d02e792fce6e03f27537848b67cce96: Status 404 returned error can't find the container with id 2f1294dfb882381793f726381e9084df1d02e792fce6e03f27537848b67cce96
Jul 24 16:17:48.835211 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:48.835167    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" event={"ID":"2c4e2241-174c-478d-84a2-958b643730bf","Type":"ContainerStarted","Data":"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"}
Jul 24 16:17:48.835407 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:48.835274    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" event={"ID":"2c4e2241-174c-478d-84a2-958b643730bf","Type":"ContainerStarted","Data":"2f1294dfb882381793f726381e9084df1d02e792fce6e03f27537848b67cce96"}
Jul 24 16:17:48.835477 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:48.835461    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:17:48.851475 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:48.851426    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" podStartSLOduration=1.851408829 podStartE2EDuration="1.851408829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:48.849854256 +0000 UTC m=+2210.402094029" watchObservedRunningTime="2026-07-24 16:17:48.851408829 +0000 UTC m=+2210.403648850"
Jul 24 16:17:50.927528 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:50.927486    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:17:51.799732 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:51.799699    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:17:53.841023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:53.840993    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:17:59.841818 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:17:59.841784    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:18:01.614738 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:18:01.614706    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:24.997934 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:18:24.997858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:29.761012 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:18:29.760976    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:18:29.862401 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:18:29.862365    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:05.587065 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:05.587024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:06.679271 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.679231    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"]
Jul 24 16:19:06.682821 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.682797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:06.685901 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.685878    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:19:06.691875 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.691847    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"]
Jul 24 16:19:06.810683 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.810638    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9\" (UID: \"d67558c4-34f5-4b0c-80e9-c8e15cdd9966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:06.911373 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.911332    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9\" (UID: \"d67558c4-34f5-4b0c-80e9-c8e15cdd9966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:06.911986 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.911967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9\" (UID: \"d67558c4-34f5-4b0c-80e9-c8e15cdd9966\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:06.994563 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:06.994524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:07.038659 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:07.038626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:07.138897 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:07.138871    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"]
Jul 24 16:19:07.141697 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:19:07.141668    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd67558c4_34f5_4b0c_80e9_c8e15cdd9966.slice/crio-7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57 WatchSource:0}: Error finding container 7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57: Status 404 returned error can't find the container with id 7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57
Jul 24 16:19:07.218425 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:07.218392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9" event={"ID":"d67558c4-34f5-4b0c-80e9-c8e15cdd9966","Type":"ContainerStarted","Data":"7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57"}
Jul 24 16:19:08.224937 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:08.224841    2567 generic.go:356] "Generic (PLEG): container finished" podID="d67558c4-34f5-4b0c-80e9-c8e15cdd9966" containerID="8b36d28adba50e169d7109905b5fe5cef3dac1921016714e11cec5882e0efffd" exitCode=0
Jul 24 16:19:08.225330 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:08.224930    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9" event={"ID":"d67558c4-34f5-4b0c-80e9-c8e15cdd9966","Type":"ContainerDied","Data":"8b36d28adba50e169d7109905b5fe5cef3dac1921016714e11cec5882e0efffd"}
Jul 24 16:19:09.365764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:09.365734    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:09.434342 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:09.434293    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca\") pod \"d67558c4-34f5-4b0c-80e9-c8e15cdd9966\" (UID: \"d67558c4-34f5-4b0c-80e9-c8e15cdd9966\") "
Jul 24 16:19:09.434726 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:09.434702    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d67558c4-34f5-4b0c-80e9-c8e15cdd9966" (UID: "d67558c4-34f5-4b0c-80e9-c8e15cdd9966"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:09.535785 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:09.535746    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d67558c4-34f5-4b0c-80e9-c8e15cdd9966-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:10.235341 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:10.235303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9" event={"ID":"d67558c4-34f5-4b0c-80e9-c8e15cdd9966","Type":"ContainerDied","Data":"7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57"}
Jul 24 16:19:10.235341 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:10.235325    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"
Jul 24 16:19:10.235724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:10.235354    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e8878a7d031c46e456fd8b61fa7c5b1b1e073992f3396363cb3eb83adc0ba57"
Jul 24 16:19:17.155010 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.154972    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:19:17.155483 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.155250    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" podUID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" containerName="maas-api" containerID="cri-o://228f21158fa56921edc243a67d29e35c6e1d45b369dc9ff6524b5e8531a632f5" gracePeriod=30
Jul 24 16:19:17.191931 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.191893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:17.260389 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.260348    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:19:17.260608 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.260574    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" podUID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" containerName="payload-pre-processing" containerID="cri-o://10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9" gracePeriod=30
Jul 24 16:19:17.276618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.276587    2567 generic.go:356] "Generic (PLEG): container finished" podID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" containerID="228f21158fa56921edc243a67d29e35c6e1d45b369dc9ff6524b5e8531a632f5" exitCode=0
Jul 24 16:19:17.276798 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.276666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" event={"ID":"ec617639-163d-4029-9b6f-6b2ef8e0ae6d","Type":"ContainerDied","Data":"228f21158fa56921edc243a67d29e35c6e1d45b369dc9ff6524b5e8531a632f5"}
Jul 24 16:19:17.306501 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.306469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:17.409911 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.409882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:17.430515 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.430488    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:19:17.506248 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.506213    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbcgf\" (UniqueName: \"kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf\") pod \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") "
Jul 24 16:19:17.506416 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.506347    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls\") pod \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") "
Jul 24 16:19:17.506416 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.506381    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca\") pod \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\" (UID: \"ec617639-163d-4029-9b6f-6b2ef8e0ae6d\") "
Jul 24 16:19:17.506790 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.506755    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec617639-163d-4029-9b6f-6b2ef8e0ae6d" (UID: "ec617639-163d-4029-9b6f-6b2ef8e0ae6d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:17.508755 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.508732    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf" (OuterVolumeSpecName: "kube-api-access-mbcgf") pod "ec617639-163d-4029-9b6f-6b2ef8e0ae6d" (UID: "ec617639-163d-4029-9b6f-6b2ef8e0ae6d"). InnerVolumeSpecName "kube-api-access-mbcgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:17.508999 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.508981    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec617639-163d-4029-9b6f-6b2ef8e0ae6d" (UID: "ec617639-163d-4029-9b6f-6b2ef8e0ae6d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:19:17.517661 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.517633    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:19:17.606947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.606903    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l24n2\" (UniqueName: \"kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2\") pod \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") "
Jul 24 16:19:17.607160 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.606971    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume\") pod \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\" (UID: \"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758\") "
Jul 24 16:19:17.607301 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.607283    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:17.607343 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.607311    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:17.607343 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.607325    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbcgf\" (UniqueName: \"kubernetes.io/projected/ec617639-163d-4029-9b6f-6b2ef8e0ae6d-kube-api-access-mbcgf\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:17.607415 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.607391    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" (UID: "a11fdf6d-b0c4-4a4d-b442-d78fdcef9758"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:17.609278 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.609256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2" (OuterVolumeSpecName: "kube-api-access-l24n2") pod "a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" (UID: "a11fdf6d-b0c4-4a4d-b442-d78fdcef9758"). InnerVolumeSpecName "kube-api-access-l24n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:17.708616 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.708523    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l24n2\" (UniqueName: \"kubernetes.io/projected/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-kube-api-access-l24n2\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:17.708616 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:17.708555    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:18.283616 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.283582    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"
Jul 24 16:19:18.284070 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.283616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d" event={"ID":"ec617639-163d-4029-9b6f-6b2ef8e0ae6d","Type":"ContainerDied","Data":"0a11cefa1503a452a2e7560bb1ae873bbbb7d9b67a5574a8a1faf27902c67581"}
Jul 24 16:19:18.284070 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.283707    2567 scope.go:117] "RemoveContainer" containerID="228f21158fa56921edc243a67d29e35c6e1d45b369dc9ff6524b5e8531a632f5"
Jul 24 16:19:18.285044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.284995    2567 generic.go:356] "Generic (PLEG): container finished" podID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" containerID="10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9" exitCode=0
Jul 24 16:19:18.285144 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.285057    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"
Jul 24 16:19:18.285213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.285057    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" event={"ID":"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758","Type":"ContainerDied","Data":"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"}
Jul 24 16:19:18.285213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.285186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t" event={"ID":"a11fdf6d-b0c4-4a4d-b442-d78fdcef9758","Type":"ContainerDied","Data":"677d8467cec1b836f76af828f89e0d56c663fad9471fce797e1963a1a90b0f52"}
Jul 24 16:19:18.294374 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.294350    2567 scope.go:117] "RemoveContainer" containerID="10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"
Jul 24 16:19:18.302736 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.302710    2567 scope.go:117] "RemoveContainer" containerID="10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"
Jul 24 16:19:18.303078 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:19:18.303054    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9\": container with ID starting with 10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9 not found: ID does not exist" containerID="10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"
Jul 24 16:19:18.303140 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.303089    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9"} err="failed to get container status \"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9\": rpc error: code = NotFound desc = could not find container \"10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9\": container with ID starting with 10be3fa559276442449718fe732bcb5c51d2389425b0589513b6c2b6c55da3f9 not found: ID does not exist"
Jul 24 16:19:18.334237 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.334206    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:19:18.339182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.339114    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-35cc2d-f7fbb8545-qbr6d"]
Jul 24 16:19:18.358588 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.358556    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:19:18.363286 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.363251    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-35cc2d-8485946f4d-frp6t"]
Jul 24 16:19:18.951092 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.951057    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" path="/var/lib/kubelet/pods/a11fdf6d-b0c4-4a4d-b442-d78fdcef9758/volumes"
Jul 24 16:19:18.951607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:18.951585    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" path="/var/lib/kubelet/pods/ec617639-163d-4029-9b6f-6b2ef8e0ae6d/volumes"
Jul 24 16:19:21.774174 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:21.774114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:21.791734 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:21.791696    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"]
Jul 24 16:19:21.794539 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:21.794509    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-35cc2d-h7tq9"]
Jul 24 16:19:22.179382 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:22.179293    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:22.949145 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:22.949091    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d67558c4-34f5-4b0c-80e9-c8e15cdd9966" path="/var/lib/kubelet/pods/d67558c4-34f5-4b0c-80e9-c8e15cdd9966/volumes"
Jul 24 16:19:28.370467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.370426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:28.470526 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.470497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:28.648408 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.648301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:28.866573 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.866542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:28.977757 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.977668    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"]
Jul 24 16:19:28.978028 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978014    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" containerName="payload-pre-processing"
Jul 24 16:19:28.978072 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978029    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" containerName="payload-pre-processing"
Jul 24 16:19:28.978072 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978043    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d67558c4-34f5-4b0c-80e9-c8e15cdd9966" containerName="revoke-keys"
Jul 24 16:19:28.978072 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978049    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d67558c4-34f5-4b0c-80e9-c8e15cdd9966" containerName="revoke-keys"
Jul 24 16:19:28.978072 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978066    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" containerName="maas-api"
Jul 24 16:19:28.978072 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978072    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" containerName="maas-api"
Jul 24 16:19:28.978252 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978141    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec617639-163d-4029-9b6f-6b2ef8e0ae6d" containerName="maas-api"
Jul 24 16:19:28.978252 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978152    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a11fdf6d-b0c4-4a4d-b442-d78fdcef9758" containerName="payload-pre-processing"
Jul 24 16:19:28.978252 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.978174    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d67558c4-34f5-4b0c-80e9-c8e15cdd9966" containerName="revoke-keys"
Jul 24 16:19:28.982663 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.982644    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:28.984999 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.984970    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:19:28.988532 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:28.988505    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"]
Jul 24 16:19:29.115433 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:29.115393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c\" (UID: \"2d48292f-a412-43ff-8ab0-6ca2e34945f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:29.216087 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:29.216046    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c\" (UID: \"2d48292f-a412-43ff-8ab0-6ca2e34945f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:29.216731 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:29.216707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c\" (UID: \"2d48292f-a412-43ff-8ab0-6ca2e34945f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:29.296198 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:29.296158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:29.444119 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:29.444080    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"]
Jul 24 16:19:29.447380 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:19:29.447343    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d48292f_a412_43ff_8ab0_6ca2e34945f5.slice/crio-c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0 WatchSource:0}: Error finding container c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0: Status 404 returned error can't find the container with id c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0
Jul 24 16:19:30.348640 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:30.348603    2567 generic.go:356] "Generic (PLEG): container finished" podID="2d48292f-a412-43ff-8ab0-6ca2e34945f5" containerID="a1d8d7bded40cf47b0b9b53f20c45136dda306bf631cc4f73c0fd558c26045f4" exitCode=0
Jul 24 16:19:30.348821 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:30.348687    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c" event={"ID":"2d48292f-a412-43ff-8ab0-6ca2e34945f5","Type":"ContainerDied","Data":"a1d8d7bded40cf47b0b9b53f20c45136dda306bf631cc4f73c0fd558c26045f4"}
Jul 24 16:19:30.348821 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:30.348738    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c" event={"ID":"2d48292f-a412-43ff-8ab0-6ca2e34945f5","Type":"ContainerStarted","Data":"c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0"}
Jul 24 16:19:31.493290 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:31.493263    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:31.638462 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:31.638362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca\") pod \"2d48292f-a412-43ff-8ab0-6ca2e34945f5\" (UID: \"2d48292f-a412-43ff-8ab0-6ca2e34945f5\") "
Jul 24 16:19:31.638753 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:31.638729    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d48292f-a412-43ff-8ab0-6ca2e34945f5" (UID: "2d48292f-a412-43ff-8ab0-6ca2e34945f5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:31.738937 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:31.738892    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d48292f-a412-43ff-8ab0-6ca2e34945f5-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:32.359689 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:32.359657    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"
Jul 24 16:19:32.359689 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:32.359671    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c" event={"ID":"2d48292f-a412-43ff-8ab0-6ca2e34945f5","Type":"ContainerDied","Data":"c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0"}
Jul 24 16:19:32.359926 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:32.359716    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c74f844f83b2d73d22821517ddfdd00689e877faf4192a09cb7184f55253d9e0"
Jul 24 16:19:39.394173 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.394121    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:19:39.394713 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.394487    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" podUID="57cae15e-ede9-4f68-810a-d0d5e704ca27" containerName="maas-api" containerID="cri-o://91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff" gracePeriod=30
Jul 24 16:19:39.423215 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.423175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:39.460270 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.460216    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:19:39.460459 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.460439    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" podUID="2c4e2241-174c-478d-84a2-958b643730bf" containerName="payload-pre-processing" containerID="cri-o://26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72" gracePeriod=30
Jul 24 16:19:39.526515 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.526480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:39.624571 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.624536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:39.707138 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.707107    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:19:39.739647 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.739622    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:19:39.803157 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume\") pod \"2c4e2241-174c-478d-84a2-958b643730bf\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") "
Jul 24 16:19:39.803334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803167    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca\") pod \"57cae15e-ede9-4f68-810a-d0d5e704ca27\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") "
Jul 24 16:19:39.803334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803223    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls\") pod \"57cae15e-ede9-4f68-810a-d0d5e704ca27\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") "
Jul 24 16:19:39.803334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803270    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zz44c\" (UniqueName: \"kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c\") pod \"2c4e2241-174c-478d-84a2-958b643730bf\" (UID: \"2c4e2241-174c-478d-84a2-958b643730bf\") "
Jul 24 16:19:39.803334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803296    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dkfz\" (UniqueName: \"kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz\") pod \"57cae15e-ede9-4f68-810a-d0d5e704ca27\" (UID: \"57cae15e-ede9-4f68-810a-d0d5e704ca27\") "
Jul 24 16:19:39.803561 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803491    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c4e2241-174c-478d-84a2-958b643730bf" (UID: "2c4e2241-174c-478d-84a2-958b643730bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:39.803615 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803565    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "57cae15e-ede9-4f68-810a-d0d5e704ca27" (UID: "57cae15e-ede9-4f68-810a-d0d5e704ca27"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:19:39.803669 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.803616    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c4e2241-174c-478d-84a2-958b643730bf-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:39.805600 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.805570    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c" (OuterVolumeSpecName: "kube-api-access-zz44c") pod "2c4e2241-174c-478d-84a2-958b643730bf" (UID: "2c4e2241-174c-478d-84a2-958b643730bf"). InnerVolumeSpecName "kube-api-access-zz44c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:39.805681 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.805622    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "57cae15e-ede9-4f68-810a-d0d5e704ca27" (UID: "57cae15e-ede9-4f68-810a-d0d5e704ca27"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:19:39.805681 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.805627    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz" (OuterVolumeSpecName: "kube-api-access-7dkfz") pod "57cae15e-ede9-4f68-810a-d0d5e704ca27" (UID: "57cae15e-ede9-4f68-810a-d0d5e704ca27"). InnerVolumeSpecName "kube-api-access-7dkfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:39.904988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.904885    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dkfz\" (UniqueName: \"kubernetes.io/projected/57cae15e-ede9-4f68-810a-d0d5e704ca27-kube-api-access-7dkfz\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:39.904988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.904933    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57cae15e-ede9-4f68-810a-d0d5e704ca27-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:39.904988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.904943    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57cae15e-ede9-4f68-810a-d0d5e704ca27-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:39.904988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:39.904953    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz44c\" (UniqueName: \"kubernetes.io/projected/2c4e2241-174c-478d-84a2-958b643730bf-kube-api-access-zz44c\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.407203 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.407169    2567 generic.go:356] "Generic (PLEG): container finished" podID="57cae15e-ede9-4f68-810a-d0d5e704ca27" containerID="91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff" exitCode=0
Jul 24 16:19:40.407660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.407207    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"
Jul 24 16:19:40.407660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.407259    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" event={"ID":"57cae15e-ede9-4f68-810a-d0d5e704ca27","Type":"ContainerDied","Data":"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"}
Jul 24 16:19:40.407660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.407324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl" event={"ID":"57cae15e-ede9-4f68-810a-d0d5e704ca27","Type":"ContainerDied","Data":"56046cb9130af619f101bb8f2980217633f6313a374b3860666110539a2f4cbc"}
Jul 24 16:19:40.407660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.407359    2567 scope.go:117] "RemoveContainer" containerID="91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"
Jul 24 16:19:40.408772 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.408753    2567 generic.go:356] "Generic (PLEG): container finished" podID="2c4e2241-174c-478d-84a2-958b643730bf" containerID="26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72" exitCode=0
Jul 24 16:19:40.408847 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.408786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" event={"ID":"2c4e2241-174c-478d-84a2-958b643730bf","Type":"ContainerDied","Data":"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"}
Jul 24 16:19:40.408847 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.408796    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"
Jul 24 16:19:40.408847 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.408826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd" event={"ID":"2c4e2241-174c-478d-84a2-958b643730bf","Type":"ContainerDied","Data":"2f1294dfb882381793f726381e9084df1d02e792fce6e03f27537848b67cce96"}
Jul 24 16:19:40.419081 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.419064    2567 scope.go:117] "RemoveContainer" containerID="91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"
Jul 24 16:19:40.419418 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:19:40.419393    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff\": container with ID starting with 91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff not found: ID does not exist" containerID="91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"
Jul 24 16:19:40.419498 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.419433    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff"} err="failed to get container status \"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff\": rpc error: code = NotFound desc = could not find container \"91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff\": container with ID starting with 91317c560be72f08dd90db47aa2a7bb28198ed69d429572b5d954ce8d6e734ff not found: ID does not exist"
Jul 24 16:19:40.419498 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.419456    2567 scope.go:117] "RemoveContainer" containerID="26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"
Jul 24 16:19:40.427582 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.427534    2567 scope.go:117] "RemoveContainer" containerID="26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"
Jul 24 16:19:40.427881 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:19:40.427856    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72\": container with ID starting with 26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72 not found: ID does not exist" containerID="26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"
Jul 24 16:19:40.428011 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.427984    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72"} err="failed to get container status \"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72\": rpc error: code = NotFound desc = could not find container \"26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72\": container with ID starting with 26e32fe64ae95ae62f927e9e7ecdf52369b0613aa8ba461eb18a38980fb6af72 not found: ID does not exist"
Jul 24 16:19:40.430175 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.430107    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:19:40.433617 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.433597    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-47a0d8-7b875d75cd-lfgbl"]
Jul 24 16:19:40.444177 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.444149    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:19:40.448171 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.448119    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-47a0d8-54f565845b-pgzxd"]
Jul 24 16:19:40.949685 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.949650    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c4e2241-174c-478d-84a2-958b643730bf" path="/var/lib/kubelet/pods/2c4e2241-174c-478d-84a2-958b643730bf/volumes"
Jul 24 16:19:40.950022 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:40.950010    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57cae15e-ede9-4f68-810a-d0d5e704ca27" path="/var/lib/kubelet/pods/57cae15e-ede9-4f68-810a-d0d5e704ca27/volumes"
Jul 24 16:19:44.058926 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:44.058883    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"]
Jul 24 16:19:44.062666 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:44.062635    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-47a0d8-jkl8c"]
Jul 24 16:19:44.066221 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:44.066187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:44.435755 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:44.435647    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:44.949094 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:44.949054    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d48292f-a412-43ff-8ab0-6ca2e34945f5" path="/var/lib/kubelet/pods/2d48292f-a412-43ff-8ab0-6ca2e34945f5/volumes"
Jul 24 16:19:50.147492 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:50.147404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:50.213039 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:50.212979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:50.412430 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:50.412330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:50.649024 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:50.648972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:50.998107 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:50.998064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:51.320567 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:51.320465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:51.484557 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:51.484514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:54.138803 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.138763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:54.225507 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225470    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:19:54.225814 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225799    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4e2241-174c-478d-84a2-958b643730bf" containerName="payload-pre-processing"
Jul 24 16:19:54.225814 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225815    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4e2241-174c-478d-84a2-958b643730bf" containerName="payload-pre-processing"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225830    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57cae15e-ede9-4f68-810a-d0d5e704ca27" containerName="maas-api"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225836    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="57cae15e-ede9-4f68-810a-d0d5e704ca27" containerName="maas-api"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225848    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d48292f-a412-43ff-8ab0-6ca2e34945f5" containerName="revoke-keys"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225853    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d48292f-a412-43ff-8ab0-6ca2e34945f5" containerName="revoke-keys"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225903    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c4e2241-174c-478d-84a2-958b643730bf" containerName="payload-pre-processing"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225911    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="57cae15e-ede9-4f68-810a-d0d5e704ca27" containerName="maas-api"
Jul 24 16:19:54.226042 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.225918    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d48292f-a412-43ff-8ab0-6ca2e34945f5" containerName="revoke-keys"
Jul 24 16:19:54.230476 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.230452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.232262 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.232214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:54.233271 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.233251    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1d6db7-serving-cert\""
Jul 24 16:19:54.240572 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.240534    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:19:54.298911 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.298877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:54.326743 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.326702    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.326954 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.326775    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.326954 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.326805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb8fw\" (UniqueName: \"kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.415996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.415916    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:54.428222 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.428190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.428299 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.428229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb8fw\" (UniqueName: \"kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.428299 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.428283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.428951 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.428934    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.430900 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.430882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.437974 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.437949    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb8fw\" (UniqueName: \"kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw\") pod \"maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.543008 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.542966    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:54.687618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.687588    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:19:54.690350 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:19:54.690318    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebdb254_02ec_445b_a225_bc376db2e656.slice/crio-baab7d0407ef5369cdec843494e18b2a015755b7b1a65dfdce28417396e5013d WatchSource:0}: Error finding container baab7d0407ef5369cdec843494e18b2a015755b7b1a65dfdce28417396e5013d: Status 404 returned error can't find the container with id baab7d0407ef5369cdec843494e18b2a015755b7b1a65dfdce28417396e5013d
Jul 24 16:19:54.691605 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.691582    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:19:54.920098 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:54.920060    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:55.018470 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.018436    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:19:55.022325 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.022296    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:55.023186 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.023171    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.025799 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.025777    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1d6db7-dockercfg-zpvrd\""
Jul 24 16:19:55.025799 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.025788    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1d6db7\""
Jul 24 16:19:55.033032 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.033006    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:19:55.055361 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.055319    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:19:55.058695 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.058670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.068785 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.068731    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:19:55.134788 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.134750    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.134947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.134826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22wtk\" (UniqueName: \"kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.134947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.134867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smhnn\" (UniqueName: \"kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.134947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.134933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.235801 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.235763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smhnn\" (UniqueName: \"kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.236317 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.235811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.236317 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.235896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.236317 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.235953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22wtk\" (UniqueName: \"kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.236617 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.236595    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.236654 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.236628    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.244323 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.244298    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22wtk\" (UniqueName: \"kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk\") pod \"payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") " pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.244455 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.244338    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smhnn\" (UniqueName: \"kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn\") pod \"payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.335041 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.334941    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:55.370947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.370912    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:55.485490 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.485430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" event={"ID":"9ebdb254-02ec-445b-a225-bc376db2e656","Type":"ContainerStarted","Data":"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"}
Jul 24 16:19:55.485662 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.485544    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" event={"ID":"9ebdb254-02ec-445b-a225-bc376db2e656","Type":"ContainerStarted","Data":"baab7d0407ef5369cdec843494e18b2a015755b7b1a65dfdce28417396e5013d"}
Jul 24 16:19:55.486073 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.486050    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:19:55.496951 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.496913    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:19:55.498627 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:19:55.498576    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0727bf_86c4_49a0_9c8e_7c8a1e66e0b7.slice/crio-e7aa8ad15de4d6c7145d4b6a3b0d79823bd18bcdd445c2d5c876dbb80d8b5067 WatchSource:0}: Error finding container e7aa8ad15de4d6c7145d4b6a3b0d79823bd18bcdd445c2d5c876dbb80d8b5067: Status 404 returned error can't find the container with id e7aa8ad15de4d6c7145d4b6a3b0d79823bd18bcdd445c2d5c876dbb80d8b5067
Jul 24 16:19:55.506168 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.506086    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" podStartSLOduration=1.079915314 podStartE2EDuration="1.506062694s" totalImagesPullingTime="426.14738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:19:55.504032626 +0000 UTC m=+2337.056272404" watchObservedRunningTime="2026-07-24 16:19:55.506062694 +0000 UTC m=+2337.058302468"
Jul 24 16:19:55.553202 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:55.548690    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:19:56.493258 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.493220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" event={"ID":"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7","Type":"ContainerStarted","Data":"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"}
Jul 24 16:19:56.493762 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.493286    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" event={"ID":"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7","Type":"ContainerStarted","Data":"e7aa8ad15de4d6c7145d4b6a3b0d79823bd18bcdd445c2d5c876dbb80d8b5067"}
Jul 24 16:19:56.493762 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.493441    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:19:56.494905 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.494877    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" event={"ID":"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007","Type":"ContainerStarted","Data":"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"}
Jul 24 16:19:56.495040 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.494966    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:19:56.495040 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.495004    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" event={"ID":"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007","Type":"ContainerStarted","Data":"2dbb6b478ccbc87492dd954160981002df4e8c9ed391afc061650574524646fe"}
Jul 24 16:19:56.511785 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.511724    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" podStartSLOduration=1.511707613 podStartE2EDuration="1.511707613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:19:56.509566055 +0000 UTC m=+2338.061805826" watchObservedRunningTime="2026-07-24 16:19:56.511707613 +0000 UTC m=+2338.063947386"
Jul 24 16:19:56.525378 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:56.525310    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" podStartSLOduration=1.525288419 podStartE2EDuration="1.525288419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:19:56.524620991 +0000 UTC m=+2338.076860764" watchObservedRunningTime="2026-07-24 16:19:56.525288419 +0000 UTC m=+2338.077528193"
Jul 24 16:19:58.613728 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:58.613679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:59.082587 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:59.082552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:59.409043 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:59.408950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:19:59.910245 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:19:59.910211    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:01.500316 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:01.500285    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:20:02.231148 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.231096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:02.303467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.303374    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:02.308651 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.308628    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.310984 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.310956    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e8e158-serving-cert\""
Jul 24 16:20:02.319263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.319241    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:02.325268 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.325236    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:02.402171 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.402117    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:02.403372 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.403349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.403444 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.403424    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.403479 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.403452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6kq9\" (UniqueName: \"kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.503981 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.503942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.503981 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.503984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6kq9\" (UniqueName: \"kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.504464 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.504056    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.504735 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.504712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.506538 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.506514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.512737 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.512712    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6kq9\" (UniqueName: \"kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9\") pod \"maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.533914 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.533887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:02.620952 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.620860    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:02.762054 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.761988    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:02.764317 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:20:02.764292    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b2d2cd_ab88_48db_a7ef_c7c22fa4551c.slice/crio-d4f5e0189350d8d1ed46e664314ef54fcbfda9c680ebf1876bb48d4f4c0c66e7 WatchSource:0}: Error finding container d4f5e0189350d8d1ed46e664314ef54fcbfda9c680ebf1876bb48d4f4c0c66e7: Status 404 returned error can't find the container with id d4f5e0189350d8d1ed46e664314ef54fcbfda9c680ebf1876bb48d4f4c0c66e7
Jul 24 16:20:02.900220 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.900116    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:02.998699 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:02.998663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:03.041229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.041187    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:03.045926 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.045907    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.048355 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.048331    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e8e158\""
Jul 24 16:20:03.048639 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.048621    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e8e158-dockercfg-ng85f\""
Jul 24 16:20:03.053960 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.053932    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:03.110365 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.110330    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.110536 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.110418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjmb5\" (UniqueName: \"kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.210909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.210880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.211044 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.210961    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjmb5\" (UniqueName: \"kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.211564 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.211545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.223078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.223050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjmb5\" (UniqueName: \"kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5\") pod \"payload-processing-e2e-api-b-e8e158-7879f59798-97fpb\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") " pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.358863 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.358832    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:03.501195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.501167    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:03.503508 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:20:03.503483    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7c6613_4c28_434a_bcca_f617a0fb9eea.slice/crio-607a6165317afb94c9b5d0c16af76936d8685bc6df78fe33728ec878141550d2 WatchSource:0}: Error finding container 607a6165317afb94c9b5d0c16af76936d8685bc6df78fe33728ec878141550d2: Status 404 returned error can't find the container with id 607a6165317afb94c9b5d0c16af76936d8685bc6df78fe33728ec878141550d2
Jul 24 16:20:03.531100 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.531055    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" event={"ID":"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c","Type":"ContainerStarted","Data":"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"}
Jul 24 16:20:03.531478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.531170    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" event={"ID":"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c","Type":"ContainerStarted","Data":"d4f5e0189350d8d1ed46e664314ef54fcbfda9c680ebf1876bb48d4f4c0c66e7"}
Jul 24 16:20:03.531478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.531221    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:03.532619 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.532594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" event={"ID":"0b7c6613-4c28-434a-bcca-f617a0fb9eea","Type":"ContainerStarted","Data":"607a6165317afb94c9b5d0c16af76936d8685bc6df78fe33728ec878141550d2"}
Jul 24 16:20:03.550886 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:03.550835    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" podStartSLOduration=1.124480771 podStartE2EDuration="1.550818573s" totalImagesPullingTime="426.337802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:03.548979229 +0000 UTC m=+2345.101219003" watchObservedRunningTime="2026-07-24 16:20:03.550818573 +0000 UTC m=+2345.103058346"
Jul 24 16:20:04.539562 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:04.539523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" event={"ID":"0b7c6613-4c28-434a-bcca-f617a0fb9eea","Type":"ContainerStarted","Data":"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"}
Jul 24 16:20:04.540032 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:04.539756    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:04.556725 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:04.556662    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" podStartSLOduration=1.556643401 podStartE2EDuration="1.556643401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:04.555033446 +0000 UTC m=+2346.107273220" watchObservedRunningTime="2026-07-24 16:20:04.556643401 +0000 UTC m=+2346.108883173"
Jul 24 16:20:06.706885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:06.706824    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:07.501682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:07.501636    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:20:07.501895 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:07.501726    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:20:08.211779 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:08.211738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:09.545530 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:09.545486    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:14.724892 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.724852    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"]
Jul 24 16:20:14.728455 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.728433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:14.730638 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.730619    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:20:14.735309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.735270    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"]
Jul 24 16:20:14.820074 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.820029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6db7-42snt\" (UID: \"3bf008d6-c67c-47e4-98fa-30f9ef901a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:14.921047 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.921007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6db7-42snt\" (UID: \"3bf008d6-c67c-47e4-98fa-30f9ef901a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:14.921709 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:14.921688    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-1d6db7-42snt\" (UID: \"3bf008d6-c67c-47e4-98fa-30f9ef901a1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:15.040783 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.040748    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:15.183626 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.183598    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"]
Jul 24 16:20:15.186336 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:20:15.186307    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf008d6_c67c_47e4_98fa_30f9ef901a1a.slice/crio-6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3 WatchSource:0}: Error finding container 6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3: Status 404 returned error can't find the container with id 6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3
Jul 24 16:20:15.546190 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.546156    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:15.607007 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.606968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt" event={"ID":"3bf008d6-c67c-47e4-98fa-30f9ef901a1a","Type":"ContainerStarted","Data":"844b823b3f2ef3748dd91c704a9e6571691bbf97dffd37055ad84803c5680202"}
Jul 24 16:20:15.607210 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.607041    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt" event={"ID":"3bf008d6-c67c-47e4-98fa-30f9ef901a1a","Type":"ContainerStarted","Data":"6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3"}
Jul 24 16:20:15.626263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:15.626212    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt" podStartSLOduration=1.626195503 podStartE2EDuration="1.626195503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:15.62431206 +0000 UTC m=+2357.176551834" watchObservedRunningTime="2026-07-24 16:20:15.626195503 +0000 UTC m=+2357.178435275"
Jul 24 16:20:16.613014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:16.612982    2567 generic.go:356] "Generic (PLEG): container finished" podID="3bf008d6-c67c-47e4-98fa-30f9ef901a1a" containerID="844b823b3f2ef3748dd91c704a9e6571691bbf97dffd37055ad84803c5680202" exitCode=0
Jul 24 16:20:16.613427 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:16.613073    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt" event={"ID":"3bf008d6-c67c-47e4-98fa-30f9ef901a1a","Type":"ContainerDied","Data":"844b823b3f2ef3748dd91c704a9e6571691bbf97dffd37055ad84803c5680202"}
Jul 24 16:20:17.742620 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:17.742591    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:17.847606 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:17.847568    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca\") pod \"3bf008d6-c67c-47e4-98fa-30f9ef901a1a\" (UID: \"3bf008d6-c67c-47e4-98fa-30f9ef901a1a\") "
Jul 24 16:20:17.847918 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:17.847896    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3bf008d6-c67c-47e4-98fa-30f9ef901a1a" (UID: "3bf008d6-c67c-47e4-98fa-30f9ef901a1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:17.948803 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:17.948724    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bf008d6-c67c-47e4-98fa-30f9ef901a1a-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:18.623697 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:18.623663    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"
Jul 24 16:20:18.623697 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:18.623686    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt" event={"ID":"3bf008d6-c67c-47e4-98fa-30f9ef901a1a","Type":"ContainerDied","Data":"6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3"}
Jul 24 16:20:18.623924 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:18.623735    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bc35ff10003b03d93918639c07e0ac344bf4b5c3e6f41bae1ae79e218b03dc3"
Jul 24 16:20:29.865441 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.865403    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:20:29.865849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.865658    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" podUID="9ebdb254-02ec-445b-a225-bc376db2e656" containerName="maas-api" containerID="cri-o://2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923" gracePeriod=30
Jul 24 16:20:29.884207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.884171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:29.910335 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.910297    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:20:29.910630 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.910576    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" podUID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" containerName="payload-processing" containerID="cri-o://4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64" gracePeriod=30
Jul 24 16:20:29.916015 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.915977    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:20:29.916414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.916350    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" podUID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" containerName="payload-pre-processing" containerID="cri-o://0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149" gracePeriod=30
Jul 24 16:20:29.951048 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:29.951003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:30.007531 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.007495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:30.185860 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.185829    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:20:30.219602 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.219575    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:20:30.223660 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.223635    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:20:30.253668 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253632    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smhnn\" (UniqueName: \"kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn\") pod \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253678    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22wtk\" (UniqueName: \"kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk\") pod \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253720    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pb8fw\" (UniqueName: \"kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw\") pod \"9ebdb254-02ec-445b-a225-bc376db2e656\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253751    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls\") pod \"9ebdb254-02ec-445b-a225-bc376db2e656\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253771    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume\") pod \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\" (UID: \"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253794    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume\") pod \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\" (UID: \"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7\") "
Jul 24 16:20:30.253885 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.253854    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca\") pod \"9ebdb254-02ec-445b-a225-bc376db2e656\" (UID: \"9ebdb254-02ec-445b-a225-bc376db2e656\") "
Jul 24 16:20:30.254319 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.254199    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" (UID: "c9ba89c8-6d1c-4f1b-884c-77e1bdb22007"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:30.254482 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.254447    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ebdb254-02ec-445b-a225-bc376db2e656" (UID: "9ebdb254-02ec-445b-a225-bc376db2e656"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:30.254482 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.254461    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" (UID: "9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:30.256747 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.256693    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk" (OuterVolumeSpecName: "kube-api-access-22wtk") pod "c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" (UID: "c9ba89c8-6d1c-4f1b-884c-77e1bdb22007"). InnerVolumeSpecName "kube-api-access-22wtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:30.257527 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.257472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn" (OuterVolumeSpecName: "kube-api-access-smhnn") pod "9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" (UID: "9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7"). InnerVolumeSpecName "kube-api-access-smhnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:30.257650 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.257623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ebdb254-02ec-445b-a225-bc376db2e656" (UID: "9ebdb254-02ec-445b-a225-bc376db2e656"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:20:30.258174 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.258147    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw" (OuterVolumeSpecName: "kube-api-access-pb8fw") pod "9ebdb254-02ec-445b-a225-bc376db2e656" (UID: "9ebdb254-02ec-445b-a225-bc376db2e656"). InnerVolumeSpecName "kube-api-access-pb8fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:30.355456 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355416    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb8fw\" (UniqueName: \"kubernetes.io/projected/9ebdb254-02ec-445b-a225-bc376db2e656-kube-api-access-pb8fw\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355456 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355446    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ebdb254-02ec-445b-a225-bc376db2e656-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355456 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355456    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355468    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355482    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ebdb254-02ec-445b-a225-bc376db2e656-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355491    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smhnn\" (UniqueName: \"kubernetes.io/projected/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7-kube-api-access-smhnn\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.355724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.355499    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22wtk\" (UniqueName: \"kubernetes.io/projected/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007-kube-api-access-22wtk\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:30.688463 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.688428    2567 generic.go:356] "Generic (PLEG): container finished" podID="9ebdb254-02ec-445b-a225-bc376db2e656" containerID="2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923" exitCode=0
Jul 24 16:20:30.688463 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.688468    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"
Jul 24 16:20:30.688719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.688510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" event={"ID":"9ebdb254-02ec-445b-a225-bc376db2e656","Type":"ContainerDied","Data":"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"}
Jul 24 16:20:30.688719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.688566    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q" event={"ID":"9ebdb254-02ec-445b-a225-bc376db2e656","Type":"ContainerDied","Data":"baab7d0407ef5369cdec843494e18b2a015755b7b1a65dfdce28417396e5013d"}
Jul 24 16:20:30.688719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.688604    2567 scope.go:117] "RemoveContainer" containerID="2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"
Jul 24 16:20:30.690120 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.690098    2567 generic.go:356] "Generic (PLEG): container finished" podID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" containerID="0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149" exitCode=0
Jul 24 16:20:30.690239 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.690141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" event={"ID":"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7","Type":"ContainerDied","Data":"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"}
Jul 24 16:20:30.690239 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.690200    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj" event={"ID":"9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7","Type":"ContainerDied","Data":"e7aa8ad15de4d6c7145d4b6a3b0d79823bd18bcdd445c2d5c876dbb80d8b5067"}
Jul 24 16:20:30.690239 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.690198    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"
Jul 24 16:20:30.691830 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.691801    2567 generic.go:356] "Generic (PLEG): container finished" podID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" containerID="4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64" exitCode=0
Jul 24 16:20:30.691971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.691841    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"
Jul 24 16:20:30.691971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.691833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" event={"ID":"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007","Type":"ContainerDied","Data":"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"}
Jul 24 16:20:30.691971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.691935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47" event={"ID":"c9ba89c8-6d1c-4f1b-884c-77e1bdb22007","Type":"ContainerDied","Data":"2dbb6b478ccbc87492dd954160981002df4e8c9ed391afc061650574524646fe"}
Jul 24 16:20:30.699537 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.699514    2567 scope.go:117] "RemoveContainer" containerID="2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"
Jul 24 16:20:30.699870 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:20:30.699846    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923\": container with ID starting with 2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923 not found: ID does not exist" containerID="2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"
Jul 24 16:20:30.699969 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.699883    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923"} err="failed to get container status \"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923\": rpc error: code = NotFound desc = could not find container \"2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923\": container with ID starting with 2c914829eb498e34d3d1bd752483950b69df412711c8d553ebd043b617887923 not found: ID does not exist"
Jul 24 16:20:30.699969 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.699907    2567 scope.go:117] "RemoveContainer" containerID="0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"
Jul 24 16:20:30.708616 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.708601    2567 scope.go:117] "RemoveContainer" containerID="0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"
Jul 24 16:20:30.708877 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:20:30.708862    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149\": container with ID starting with 0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149 not found: ID does not exist" containerID="0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"
Jul 24 16:20:30.708918 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.708885    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149"} err="failed to get container status \"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149\": rpc error: code = NotFound desc = could not find container \"0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149\": container with ID starting with 0c4d277e8f38632dd4f3b09454d815ff5797d0d9217bdfe3f0ac9064cb021149 not found: ID does not exist"
Jul 24 16:20:30.708918 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.708900    2567 scope.go:117] "RemoveContainer" containerID="4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"
Jul 24 16:20:30.714631 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.714598    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:20:30.717144 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.717093    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1d6db7-79b69fd7bd-zp47q"]
Jul 24 16:20:30.718064 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.718050    2567 scope.go:117] "RemoveContainer" containerID="4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"
Jul 24 16:20:30.718479 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:20:30.718460    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64\": container with ID starting with 4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64 not found: ID does not exist" containerID="4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"
Jul 24 16:20:30.718526 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.718487    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64"} err="failed to get container status \"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64\": rpc error: code = NotFound desc = could not find container \"4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64\": container with ID starting with 4806a20f280e87780fa0f24796db81a775bee395e0e196b65011b6d0459dcc64 not found: ID does not exist"
Jul 24 16:20:30.727931 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.727905    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:20:30.731730 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.731710    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1d6db7-6c6f95946d-hhf47"]
Jul 24 16:20:30.744198 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.744172    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:20:30.748483 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.748458    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-1d6db7-9fbc57c44-lrzjj"]
Jul 24 16:20:30.950612 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.950527    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" path="/var/lib/kubelet/pods/9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7/volumes"
Jul 24 16:20:30.951008 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.950899    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ebdb254-02ec-445b-a225-bc376db2e656" path="/var/lib/kubelet/pods/9ebdb254-02ec-445b-a225-bc376db2e656/volumes"
Jul 24 16:20:30.951218 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:30.951203    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" path="/var/lib/kubelet/pods/c9ba89c8-6d1c-4f1b-884c-77e1bdb22007/volumes"
Jul 24 16:20:34.809213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:34.809160    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"]
Jul 24 16:20:34.811111 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:34.811085    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-1d6db7-42snt"]
Jul 24 16:20:34.811858 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:34.811834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:34.951786 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:34.951756    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bf008d6-c67c-47e4-98fa-30f9ef901a1a" path="/var/lib/kubelet/pods/3bf008d6-c67c-47e4-98fa-30f9ef901a1a/volumes"
Jul 24 16:20:35.176695 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:35.176591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.201269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.201233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.303541 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.303511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.434769 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.434728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.672146 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.672092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:41.793461 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.793418    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"]
Jul 24 16:20:41.793956 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.793937    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ebdb254-02ec-445b-a225-bc376db2e656" containerName="maas-api"
Jul 24 16:20:41.793996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.793961    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ebdb254-02ec-445b-a225-bc376db2e656" containerName="maas-api"
Jul 24 16:20:41.793996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.793979    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bf008d6-c67c-47e4-98fa-30f9ef901a1a" containerName="revoke-keys"
Jul 24 16:20:41.793996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.793989    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bf008d6-c67c-47e4-98fa-30f9ef901a1a" containerName="revoke-keys"
Jul 24 16:20:41.794088 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794018    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" containerName="payload-processing"
Jul 24 16:20:41.794088 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794027    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" containerName="payload-processing"
Jul 24 16:20:41.794088 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794043    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" containerName="payload-pre-processing"
Jul 24 16:20:41.794088 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794054    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" containerName="payload-pre-processing"
Jul 24 16:20:41.794231 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794201    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ebdb254-02ec-445b-a225-bc376db2e656" containerName="maas-api"
Jul 24 16:20:41.794231 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794217    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a0727bf-86c4-49a0-9c8e-7c8a1e66e0b7" containerName="payload-pre-processing"
Jul 24 16:20:41.794231 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794224    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9ba89c8-6d1c-4f1b-884c-77e1bdb22007" containerName="payload-processing"
Jul 24 16:20:41.794318 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.794234    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bf008d6-c67c-47e4-98fa-30f9ef901a1a" containerName="revoke-keys"
Jul 24 16:20:41.797470 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.797442    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:41.800079 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.800048    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:20:41.803587 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.803562    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"]
Jul 24 16:20:41.860383 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.860341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw\" (UID: \"5ef4c616-bb60-449a-9267-e97887b76097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:41.961292 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.961189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw\" (UID: \"5ef4c616-bb60-449a-9267-e97887b76097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:41.961881 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:41.961859    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw\" (UID: \"5ef4c616-bb60-449a-9267-e97887b76097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:42.109699 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:42.109660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:42.261314 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:42.261286    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"]
Jul 24 16:20:42.263734 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:20:42.263702    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef4c616_bb60_449a_9267_e97887b76097.slice/crio-e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0 WatchSource:0}: Error finding container e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0: Status 404 returned error can't find the container with id e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0
Jul 24 16:20:42.753788 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:42.753755    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw" event={"ID":"5ef4c616-bb60-449a-9267-e97887b76097","Type":"ContainerStarted","Data":"a18f457c9a1ef82404b161fb4097ec349ba346606f4e6ab4585937a6d1155a6c"}
Jul 24 16:20:42.754013 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:42.753815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw" event={"ID":"5ef4c616-bb60-449a-9267-e97887b76097","Type":"ContainerStarted","Data":"e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0"}
Jul 24 16:20:42.770090 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:42.770025    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw" podStartSLOduration=1.770008187 podStartE2EDuration="1.770008187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:42.768311226 +0000 UTC m=+2384.320551003" watchObservedRunningTime="2026-07-24 16:20:42.770008187 +0000 UTC m=+2384.322248010"
Jul 24 16:20:43.759795 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:43.759764    2567 generic.go:356] "Generic (PLEG): container finished" podID="5ef4c616-bb60-449a-9267-e97887b76097" containerID="a18f457c9a1ef82404b161fb4097ec349ba346606f4e6ab4585937a6d1155a6c" exitCode=0
Jul 24 16:20:43.760215 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:43.759869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw" event={"ID":"5ef4c616-bb60-449a-9267-e97887b76097","Type":"ContainerDied","Data":"a18f457c9a1ef82404b161fb4097ec349ba346606f4e6ab4585937a6d1155a6c"}
Jul 24 16:20:44.907470 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:44.907421    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:44.984863 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:44.984826    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca\") pod \"5ef4c616-bb60-449a-9267-e97887b76097\" (UID: \"5ef4c616-bb60-449a-9267-e97887b76097\") "
Jul 24 16:20:44.985235 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:44.985211    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ef4c616-bb60-449a-9267-e97887b76097" (UID: "5ef4c616-bb60-449a-9267-e97887b76097"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:45.086046 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:45.085944    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef4c616-bb60-449a-9267-e97887b76097-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:45.770942 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:45.770905    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"
Jul 24 16:20:45.770942 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:45.770916    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw" event={"ID":"5ef4c616-bb60-449a-9267-e97887b76097","Type":"ContainerDied","Data":"e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0"}
Jul 24 16:20:45.771193 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:45.770971    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8cfe2da283c87182a737eb869e139884708267a674c702525548db07593dbc0"
Jul 24 16:20:57.281634 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.281578    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:57.282263 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.281852    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" podUID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" containerName="maas-api" containerID="cri-o://f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68" gracePeriod=30
Jul 24 16:20:57.309178 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.309118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:57.333037 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.333005    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:57.333277 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.333250    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" podUID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" containerName="payload-processing" containerID="cri-o://4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b" gracePeriod=30
Jul 24 16:20:57.407040 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.407005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:57.466257 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.466224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:20:57.580332 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.580308    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:57.611601 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.611571    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:57.701391 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701354    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls\") pod \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") "
Jul 24 16:20:57.701391 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701397    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume\") pod \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") "
Jul 24 16:20:57.701682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701432    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6kq9\" (UniqueName: \"kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9\") pod \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") "
Jul 24 16:20:57.701682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701458    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca\") pod \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\" (UID: \"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c\") "
Jul 24 16:20:57.701682 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701482    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjmb5\" (UniqueName: \"kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5\") pod \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\" (UID: \"0b7c6613-4c28-434a-bcca-f617a0fb9eea\") "
Jul 24 16:20:57.701894 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701858    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" (UID: "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:57.701894 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.701877    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b7c6613-4c28-434a-bcca-f617a0fb9eea" (UID: "0b7c6613-4c28-434a-bcca-f617a0fb9eea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:57.703808 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.703772    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" (UID: "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:20:57.703808 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.703781    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9" (OuterVolumeSpecName: "kube-api-access-v6kq9") pod "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" (UID: "f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c"). InnerVolumeSpecName "kube-api-access-v6kq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:57.703947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.703810    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5" (OuterVolumeSpecName: "kube-api-access-mjmb5") pod "0b7c6613-4c28-434a-bcca-f617a0fb9eea" (UID: "0b7c6613-4c28-434a-bcca-f617a0fb9eea"). InnerVolumeSpecName "kube-api-access-mjmb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:57.802400 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.802308    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:57.802400 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.802344    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b7c6613-4c28-434a-bcca-f617a0fb9eea-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:57.802400 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.802355    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6kq9\" (UniqueName: \"kubernetes.io/projected/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-kube-api-access-v6kq9\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:57.802400 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.802364    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:57.802400 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.802372    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjmb5\" (UniqueName: \"kubernetes.io/projected/0b7c6613-4c28-434a-bcca-f617a0fb9eea-kube-api-access-mjmb5\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:20:57.832940 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.832902    2567 generic.go:356] "Generic (PLEG): container finished" podID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" containerID="f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68" exitCode=0
Jul 24 16:20:57.833119 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.832957    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"
Jul 24 16:20:57.833119 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.832988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" event={"ID":"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c","Type":"ContainerDied","Data":"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"}
Jul 24 16:20:57.833119 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.833045    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t" event={"ID":"f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c","Type":"ContainerDied","Data":"d4f5e0189350d8d1ed46e664314ef54fcbfda9c680ebf1876bb48d4f4c0c66e7"}
Jul 24 16:20:57.833119 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.833072    2567 scope.go:117] "RemoveContainer" containerID="f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"
Jul 24 16:20:57.834664 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.834633    2567 generic.go:356] "Generic (PLEG): container finished" podID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" containerID="4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b" exitCode=0
Jul 24 16:20:57.834664 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.834660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" event={"ID":"0b7c6613-4c28-434a-bcca-f617a0fb9eea","Type":"ContainerDied","Data":"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"}
Jul 24 16:20:57.834825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.834693    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"
Jul 24 16:20:57.834825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.834722    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb" event={"ID":"0b7c6613-4c28-434a-bcca-f617a0fb9eea","Type":"ContainerDied","Data":"607a6165317afb94c9b5d0c16af76936d8685bc6df78fe33728ec878141550d2"}
Jul 24 16:20:57.846253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.845282    2567 scope.go:117] "RemoveContainer" containerID="f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"
Jul 24 16:20:57.846253 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:20:57.845646    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68\": container with ID starting with f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68 not found: ID does not exist" containerID="f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"
Jul 24 16:20:57.846253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.845678    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68"} err="failed to get container status \"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68\": rpc error: code = NotFound desc = could not find container \"f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68\": container with ID starting with f5141e418b3042f9e939d0f4027075f9c4b2771e6c85bb073608e1cc04c61f68 not found: ID does not exist"
Jul 24 16:20:57.846253 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.845705    2567 scope.go:117] "RemoveContainer" containerID="4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"
Jul 24 16:20:57.856629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.856608    2567 scope.go:117] "RemoveContainer" containerID="4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"
Jul 24 16:20:57.856937 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:20:57.856914    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b\": container with ID starting with 4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b not found: ID does not exist" containerID="4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"
Jul 24 16:20:57.857025 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.856942    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b"} err="failed to get container status \"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b\": rpc error: code = NotFound desc = could not find container \"4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b\": container with ID starting with 4fb1f6f7cf3d2b2007931015504e83ab6104f8c96b23a4d5ee836d3c081bbc3b not found: ID does not exist"
Jul 24 16:20:57.859749 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.859728    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:57.863764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.863743    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e8e158-6bc9dd8c7-zkd7t"]
Jul 24 16:20:57.873759 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.873734    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:57.877918 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:57.877898    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e8e158-7879f59798-97fpb"]
Jul 24 16:20:58.949834 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:58.949802    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" path="/var/lib/kubelet/pods/0b7c6613-4c28-434a-bcca-f617a0fb9eea/volumes"
Jul 24 16:20:58.950343 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:20:58.950323    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" path="/var/lib/kubelet/pods/f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c/volumes"
Jul 24 16:21:01.884298 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:01.884259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:01.887949 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:01.887914    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"]
Jul 24 16:21:01.888209 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:01.888192    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e8e158-rnjrw"]
Jul 24 16:21:02.266642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:02.266597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:02.949052 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:02.949017    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ef4c616-bb60-449a-9267-e97887b76097" path="/var/lib/kubelet/pods/5ef4c616-bb60-449a-9267-e97887b76097/volumes"
Jul 24 16:21:08.607505 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:08.607465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:08.706391 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:08.706352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:08.877423 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:08.877337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:09.111266 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:09.111209    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:09.488976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:09.488928    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:09.824777 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:09.824653    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:10.479466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:10.479430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:12.700012 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.699951    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:12.792405 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.792362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:12.935698 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.935668    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:21:12.935988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.935977    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ef4c616-bb60-449a-9267-e97887b76097" containerName="revoke-keys"
Jul 24 16:21:12.936036 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.935990    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ef4c616-bb60-449a-9267-e97887b76097" containerName="revoke-keys"
Jul 24 16:21:12.936036 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936004    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" containerName="maas-api"
Jul 24 16:21:12.936036 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936010    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" containerName="maas-api"
Jul 24 16:21:12.936036 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936025    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" containerName="payload-processing"
Jul 24 16:21:12.936036 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936031    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" containerName="payload-processing"
Jul 24 16:21:12.936216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936083    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b2d2cd-ab88-48db-a7ef-c7c22fa4551c" containerName="maas-api"
Jul 24 16:21:12.936216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936091    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b7c6613-4c28-434a-bcca-f617a0fb9eea" containerName="payload-processing"
Jul 24 16:21:12.936216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.936098    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ef4c616-bb60-449a-9267-e97887b76097" containerName="revoke-keys"
Jul 24 16:21:12.939064 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.939044    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:12.941675 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.941651    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-ed0643-serving-cert\""
Jul 24 16:21:12.952788 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.952729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:21:12.966592 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:12.966556    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:13.042849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.042816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.042849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.042863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.043088 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.042963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdp7f\" (UniqueName: \"kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.144245 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.144160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdp7f\" (UniqueName: \"kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.144399 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.144291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.144399 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.144337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.144488 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:13.144445    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-serving-cert: secret "maas-api-e2e-shared-a-ed0643-serving-cert" not found
Jul 24 16:21:13.144530 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:13.144519    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls podName:6e2f08a9-259d-4fab-85be-d5043c97602e nodeName:}" failed. No retries permitted until 2026-07-24 16:21:13.644495852 +0000 UTC m=+2415.196735618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls") pod "maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" (UID: "6e2f08a9-259d-4fab-85be-d5043c97602e") : secret "maas-api-e2e-shared-a-ed0643-serving-cert" not found
Jul 24 16:21:13.145162 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.145113    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.153737 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.153701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdp7f\" (UniqueName: \"kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.443166 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.443119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:13.545808 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.545773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:13.557747 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.557712    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:21:13.561265 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.561243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.563864 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.563838    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-ed0643\""
Jul 24 16:21:13.564023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.563866    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-ed0643-dockercfg-s4x5p\""
Jul 24 16:21:13.570248 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.570216    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:21:13.590739 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.590700    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:21:13.594657 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.594633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.606076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.606044    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:21:13.648849 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.648809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:13.649114 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:13.649024    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-serving-cert: secret "maas-api-e2e-shared-a-ed0643-serving-cert" not found
Jul 24 16:21:13.649114 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:13.649099    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls podName:6e2f08a9-259d-4fab-85be-d5043c97602e nodeName:}" failed. No retries permitted until 2026-07-24 16:21:14.649079303 +0000 UTC m=+2416.201319061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls") pod "maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" (UID: "6e2f08a9-259d-4fab-85be-d5043c97602e") : secret "maas-api-e2e-shared-a-ed0643-serving-cert" not found
Jul 24 16:21:13.749390 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.749350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.749390 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.749391    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbv4\" (UniqueName: \"kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.749843 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.749421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkblb\" (UniqueName: \"kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.749843 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.749578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.788793 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.788752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:13.851022 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.850978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.851237 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.851036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.851237 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.851065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djbv4\" (UniqueName: \"kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.851237 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.851106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkblb\" (UniqueName: \"kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.851927 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.851901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.851985 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.851901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.859654 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.859621    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkblb\" (UniqueName: \"kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb\") pod \"payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") " pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:13.859771 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.859716    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbv4\" (UniqueName: \"kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4\") pod \"payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.873612 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.873580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:13.907598 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:13.907561    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:14.041419 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.041379    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:21:14.045968 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:21:14.045923    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db665e4_83c7_4a69_b688_4af5a1b66996.slice/crio-21f3ee4e1ddcb6e16df329702664521d87354026eacb7c5ea301691176088a2b WatchSource:0}: Error finding container 21f3ee4e1ddcb6e16df329702664521d87354026eacb7c5ea301691176088a2b: Status 404 returned error can't find the container with id 21f3ee4e1ddcb6e16df329702664521d87354026eacb7c5ea301691176088a2b
Jul 24 16:21:14.068334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.068303    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:21:14.070205 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:21:14.070178    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f7a349_7926_4781_ad00_bd9bb04f4d25.slice/crio-b46fd8a8bbfcae2b7ae44866bf518454cf2353d36dfd7b9d52cc5c89a63c4bee WatchSource:0}: Error finding container b46fd8a8bbfcae2b7ae44866bf518454cf2353d36dfd7b9d52cc5c89a63c4bee: Status 404 returned error can't find the container with id b46fd8a8bbfcae2b7ae44866bf518454cf2353d36dfd7b9d52cc5c89a63c4bee
Jul 24 16:21:14.660262 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.660219    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:14.662891 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.662861    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:14.751758 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.751709    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:14.898160 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.898066    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:21:14.902179 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:21:14.902152    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2f08a9_259d_4fab_85be_d5043c97602e.slice/crio-a1e8640bf83ca18f8f7efca8b623730091126e531a9ecf1bf5c5e28bbb2d12c9 WatchSource:0}: Error finding container a1e8640bf83ca18f8f7efca8b623730091126e531a9ecf1bf5c5e28bbb2d12c9: Status 404 returned error can't find the container with id a1e8640bf83ca18f8f7efca8b623730091126e531a9ecf1bf5c5e28bbb2d12c9
Jul 24 16:21:14.921621 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.921519    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" event={"ID":"7db665e4-83c7-4a69-b688-4af5a1b66996","Type":"ContainerStarted","Data":"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"}
Jul 24 16:21:14.921621 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.921586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:14.921621 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.921611    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" event={"ID":"7db665e4-83c7-4a69-b688-4af5a1b66996","Type":"ContainerStarted","Data":"21f3ee4e1ddcb6e16df329702664521d87354026eacb7c5ea301691176088a2b"}
Jul 24 16:21:14.923015 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.922988    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" event={"ID":"6e2f08a9-259d-4fab-85be-d5043c97602e","Type":"ContainerStarted","Data":"a1e8640bf83ca18f8f7efca8b623730091126e531a9ecf1bf5c5e28bbb2d12c9"}
Jul 24 16:21:14.924746 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.924721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" event={"ID":"20f7a349-7926-4781-ad00-bd9bb04f4d25","Type":"ContainerStarted","Data":"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"}
Jul 24 16:21:14.924891 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.924766    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" event={"ID":"20f7a349-7926-4781-ad00-bd9bb04f4d25","Type":"ContainerStarted","Data":"b46fd8a8bbfcae2b7ae44866bf518454cf2353d36dfd7b9d52cc5c89a63c4bee"}
Jul 24 16:21:14.924891 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.924792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:14.937093 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.937033    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" podStartSLOduration=1.9370119369999999 podStartE2EDuration="1.937011937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:14.9366756 +0000 UTC m=+2416.488915373" watchObservedRunningTime="2026-07-24 16:21:14.937011937 +0000 UTC m=+2416.489251711"
Jul 24 16:21:14.953497 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:14.953441    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" podStartSLOduration=1.953420902 podStartE2EDuration="1.953420902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:14.951820916 +0000 UTC m=+2416.504060691" watchObservedRunningTime="2026-07-24 16:21:14.953420902 +0000 UTC m=+2416.505660675"
Jul 24 16:21:15.931351 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:15.931309    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" event={"ID":"6e2f08a9-259d-4fab-85be-d5043c97602e","Type":"ContainerStarted","Data":"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"}
Jul 24 16:21:15.952205 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:15.952120    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" podStartSLOduration=3.442508224 podStartE2EDuration="3.952104898s" totalImagesPullingTime="509.596674ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:15.949815127 +0000 UTC m=+2417.502054911" watchObservedRunningTime="2026-07-24 16:21:15.952104898 +0000 UTC m=+2417.504344671"
Jul 24 16:21:16.936302 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:16.936267    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:17.258834 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:17.258798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:17.594104 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:17.594018    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:17.917078 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:17.916979    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:18.161269 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:18.161233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:20.729350 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.729256    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:20.829117 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.829075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:20.950085 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.950047    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:21:20.953807 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.953782    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:20.958547 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.958518    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-5aebfb-serving-cert\""
Jul 24 16:21:20.967115 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.967083    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:21:20.986295 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:20.986265    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:21.015791 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.015748    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.015791 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.015792    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd6b7\" (UniqueName: \"kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.016049 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.015889    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.117066 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.117021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.117339 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.117141    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.117339 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.117176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd6b7\" (UniqueName: \"kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.117463 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:21.117331    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-serving-cert: secret "maas-api-e2e-shared-b-5aebfb-serving-cert" not found
Jul 24 16:21:21.117463 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:21:21.117418    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls podName:c232c7e4-6193-48d3-8bd3-c978a9a3d73d nodeName:}" failed. No retries permitted until 2026-07-24 16:21:21.617394375 +0000 UTC m=+2423.169634130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls") pod "maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" (UID: "c232c7e4-6193-48d3-8bd3-c978a9a3d73d") : secret "maas-api-e2e-shared-b-5aebfb-serving-cert" not found
Jul 24 16:21:21.117840 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.117818    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.126800 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.126762    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd6b7\" (UniqueName: \"kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.289066 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.288981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:21.622048 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.621949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.624732 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.624701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") pod \"maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.788038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.788003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:21.865751 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.865715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:21.896996 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.896952    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:21.985402 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.985358    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:21:21.989977 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.989940    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:21.992956 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.992832    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-5aebfb\""
Jul 24 16:21:21.992956 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:21.992880    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-5aebfb-dockercfg-sg45b\""
Jul 24 16:21:22.004494 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.004462    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:21:22.025593 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.025538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.025593 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.025593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpfwd\" (UniqueName: \"kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.038205 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.038177    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:21:22.041099 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:21:22.041067    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc232c7e4_6193_48d3_8bd3_c978a9a3d73d.slice/crio-97b4295073433e8a649b8b3f605a2e45f1891ea3010cda1153f000d60a37d500 WatchSource:0}: Error finding container 97b4295073433e8a649b8b3f605a2e45f1891ea3010cda1153f000d60a37d500: Status 404 returned error can't find the container with id 97b4295073433e8a649b8b3f605a2e45f1891ea3010cda1153f000d60a37d500
Jul 24 16:21:22.126523 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.126481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.126523 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.126534    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpfwd\" (UniqueName: \"kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.127225 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.127203    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.153232 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.153158    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpfwd\" (UniqueName: \"kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd\") pod \"payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.306151 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.306083    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.462526 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.462498    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:21:22.465042 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:21:22.464995    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ebcf41_28fa_4df5_9636_c5c129209c46.slice/crio-8d74c22fbbff066bd14e0a34a9903b5ed1121f5d78a139269228960bafa51a1b WatchSource:0}: Error finding container 8d74c22fbbff066bd14e0a34a9903b5ed1121f5d78a139269228960bafa51a1b: Status 404 returned error can't find the container with id 8d74c22fbbff066bd14e0a34a9903b5ed1121f5d78a139269228960bafa51a1b
Jul 24 16:21:22.954636 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.954608    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:21:22.983052 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.983000    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" event={"ID":"75ebcf41-28fa-4df5-9636-c5c129209c46","Type":"ContainerStarted","Data":"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"}
Jul 24 16:21:22.983262 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.983095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" event={"ID":"75ebcf41-28fa-4df5-9636-c5c129209c46","Type":"ContainerStarted","Data":"8d74c22fbbff066bd14e0a34a9903b5ed1121f5d78a139269228960bafa51a1b"}
Jul 24 16:21:22.983262 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.983145    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:22.985627 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.985596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" event={"ID":"c232c7e4-6193-48d3-8bd3-c978a9a3d73d","Type":"ContainerStarted","Data":"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"}
Jul 24 16:21:22.985754 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.985643    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" event={"ID":"c232c7e4-6193-48d3-8bd3-c978a9a3d73d","Type":"ContainerStarted","Data":"97b4295073433e8a649b8b3f605a2e45f1891ea3010cda1153f000d60a37d500"}
Jul 24 16:21:22.985754 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:22.985696    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:23.025499 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:23.025419    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" podStartSLOduration=2.025353429 podStartE2EDuration="2.025353429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:23.02236281 +0000 UTC m=+2424.574602584" watchObservedRunningTime="2026-07-24 16:21:23.025353429 +0000 UTC m=+2424.577593202"
Jul 24 16:21:23.053439 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:23.053368    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" podStartSLOduration=2.737471136 podStartE2EDuration="3.053345777s" totalImagesPullingTime="315.874641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:23.052146275 +0000 UTC m=+2424.604386042" watchObservedRunningTime="2026-07-24 16:21:23.053345777 +0000 UTC m=+2424.605585551"
Jul 24 16:21:25.216407 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:25.216366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:25.933727 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:25.933699    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:21:25.933921 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:25.933776    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:21:28.996395 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:28.996367    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:21:33.992562 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:33.992526    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:21:46.102778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:46.102742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:21:59.182764 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:59.182731    2567 scope.go:117] "RemoveContainer" containerID="2924afb3bd1c864052b187fb2578b9417c2895332c72caaf206c75ffb5aa0f6d"
Jul 24 16:21:59.193156 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:21:59.193112    2567 scope.go:117] "RemoveContainer" containerID="4a91c608c3215d1b7f415d50e1ae4d298c646355814a9c87ee5ad1682ff882bd"
Jul 24 16:22:28.306390 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:22:28.306341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:50.999377 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:22:50.999252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:51.101219 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:22:51.101179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:22:56.297216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:22:56.297179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:31.625532 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:23:31.625480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:34.225868 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:23:34.225830    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:23:43.014460 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:23:43.014425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:16.705642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:24:16.705601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:17.750255 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:24:17.750213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:43.384560 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:24:43.384471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:44.398144 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:24:44.398080    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:24:49.397103 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:24:49.397061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:19.260765 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:19.260730    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:29.294581 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:29.294544    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:54.432315 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:54.432224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:54.533280 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:54.533246    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:25:59.237782 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:59.237745    2567 scope.go:117] "RemoveContainer" containerID="a1d8d7bded40cf47b0b9b53f20c45136dda306bf631cc4f73c0fd558c26045f4"
Jul 24 16:25:59.247466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:25:59.247440    2567 scope.go:117] "RemoveContainer" containerID="8b36d28adba50e169d7109905b5fe5cef3dac1921016714e11cec5882e0efffd"
Jul 24 16:26:13.659706 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:26:13.659657    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:14.566458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:26:14.566420    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:31.983292 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:26:31.983260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:26:59.275958 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:26:59.275918    2567 scope.go:117] "RemoveContainer" containerID="a18f457c9a1ef82404b161fb4097ec349ba346606f4e6ab4585937a6d1155a6c"
Jul 24 16:26:59.284591 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:26:59.284566    2567 scope.go:117] "RemoveContainer" containerID="844b823b3f2ef3748dd91c704a9e6571691bbf97dffd37055ad84803c5680202"
Jul 24 16:27:11.301607 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:27:11.301565    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:48.112115 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:27:48.112081    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:27:49.076457 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:27:49.076419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:09.040891 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:09.040854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:09.317440 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:09.317340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:09.542207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:09.542159    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:11.939550 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:11.939508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.017173 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.017102    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:28:12.021157 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.021117    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.023801 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.023778    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2c5dfe-serving-cert\""
Jul 24 16:28:12.030276 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.030251    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:28:12.036196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.036173    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.071458 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.071425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrb6\" (UniqueName: \"kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.071665 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.071493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.071665 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.071515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.133863 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.133810    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.172282 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.172243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrb6\" (UniqueName: \"kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.172494 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.172312    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.172494 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.172331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.172494 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:28:12.172420    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-serving-cert: secret "maas-api-e2e-ipp-a-2c5dfe-serving-cert" not found
Jul 24 16:28:12.172666 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:28:12.172518    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls podName:27344a50-35c4-4893-b4d9-7f64d64b296d nodeName:}" failed. No retries permitted until 2026-07-24 16:28:12.67249574 +0000 UTC m=+2834.224735496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls") pod "maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" (UID: "27344a50-35c4-4893-b4d9-7f64d64b296d") : secret "maas-api-e2e-ipp-a-2c5dfe-serving-cert" not found
Jul 24 16:28:12.172893 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.172875    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.183937 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.183908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrb6\" (UniqueName: \"kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.426396 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.426358    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.677532 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.677443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.680309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.680277    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.686848 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.686808    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.762845 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.762806    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:28:12.766382 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.766358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.768908 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.768884    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-2c5dfe-dockercfg-fzllc\""
Jul 24 16:28:12.769074 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.769035    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-2c5dfe\""
Jul 24 16:28:12.775779 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.775749    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:28:12.776473 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.776443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:12.880320 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.880273    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prf4q\" (UniqueName: \"kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.880513 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.880370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.933962 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.933856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:12.981053 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.981018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prf4q\" (UniqueName: \"kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.981612 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.981098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.982033 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.982010    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:12.989390 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:12.989360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prf4q\" (UniqueName: \"kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q\") pod \"payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:13.080662 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.080623    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:13.099992 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.099765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:28:13.102652 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:28:13.102615    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27344a50_35c4_4893_b4d9_7f64d64b296d.slice/crio-c24d5900e25cf1bada3e027cf5400410ec34d2f083daf75828fbfe217d028543 WatchSource:0}: Error finding container c24d5900e25cf1bada3e027cf5400410ec34d2f083daf75828fbfe217d028543: Status 404 returned error can't find the container with id c24d5900e25cf1bada3e027cf5400410ec34d2f083daf75828fbfe217d028543
Jul 24 16:28:13.104015 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.103992    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:28:13.233956 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.233926    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:28:13.236365 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:28:13.236299    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47071e4a_ff13_4795_90a3_4af33734328f.slice/crio-26ccb7048143c80b1ce0e860cf4ba87a89d7e465edea19ba44ddcff64ae7bb43 WatchSource:0}: Error finding container 26ccb7048143c80b1ce0e860cf4ba87a89d7e465edea19ba44ddcff64ae7bb43: Status 404 returned error can't find the container with id 26ccb7048143c80b1ce0e860cf4ba87a89d7e465edea19ba44ddcff64ae7bb43
Jul 24 16:28:13.990110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.989807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" event={"ID":"47071e4a-ff13-4795-90a3-4af33734328f","Type":"ContainerStarted","Data":"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"}
Jul 24 16:28:13.990110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.989878    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" event={"ID":"47071e4a-ff13-4795-90a3-4af33734328f","Type":"ContainerStarted","Data":"26ccb7048143c80b1ce0e860cf4ba87a89d7e465edea19ba44ddcff64ae7bb43"}
Jul 24 16:28:13.990110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.989913    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:13.991939 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.991901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" event={"ID":"27344a50-35c4-4893-b4d9-7f64d64b296d","Type":"ContainerStarted","Data":"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"}
Jul 24 16:28:13.992104 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.991956    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" event={"ID":"27344a50-35c4-4893-b4d9-7f64d64b296d","Type":"ContainerStarted","Data":"c24d5900e25cf1bada3e027cf5400410ec34d2f083daf75828fbfe217d028543"}
Jul 24 16:28:13.992104 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:13.992018    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:14.005369 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:14.005297    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" podStartSLOduration=2.005278342 podStartE2EDuration="2.005278342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:14.004035731 +0000 UTC m=+2835.556275507" watchObservedRunningTime="2026-07-24 16:28:14.005278342 +0000 UTC m=+2835.557518114"
Jul 24 16:28:14.020971 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:14.020914    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" podStartSLOduration=2.518999446 podStartE2EDuration="3.020893937s" totalImagesPullingTime="501.894491ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:14.019727991 +0000 UTC m=+2835.571967765" watchObservedRunningTime="2026-07-24 16:28:14.020893937 +0000 UTC m=+2835.573133710"
Jul 24 16:28:16.442396 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:16.442361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:20.002923 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:20.002893    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:28:24.998955 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:24.998921    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:28:27.381492 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:27.381452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:27.685209 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:27.685076    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:27.811835 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:27.811799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:33.382637 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.382604    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:33.476082 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.476036    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:33.819813 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.819764    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:28:33.823724 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.823691    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:33.829405 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.829373    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-2c8631-serving-cert\""
Jul 24 16:28:33.842247 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.842207    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:28:33.865980 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.865938    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:33.983469 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.983425    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cqn\" (UniqueName: \"kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:33.983671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.983509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:33.983671 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.983550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:33.990091 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:33.990053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:34.084603 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.084498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cqn\" (UniqueName: \"kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.084603 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.084569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.084802 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.084621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.085261 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.085225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.087365 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.087333    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.096612 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.096584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cqn\" (UniqueName: \"kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn\") pod \"maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.135291 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.135255    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:34.287377 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.287347    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:28:34.289571 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:28:34.289529    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7c834d_1b78_4bcd_a32b_23501fb1d61f.slice/crio-024a3736dbefc1b764027329645154df89da33eb33059ea7e454f37ecc5532c6 WatchSource:0}: Error finding container 024a3736dbefc1b764027329645154df89da33eb33059ea7e454f37ecc5532c6: Status 404 returned error can't find the container with id 024a3736dbefc1b764027329645154df89da33eb33059ea7e454f37ecc5532c6
Jul 24 16:28:34.378016 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.377923    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:34.462379 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.462344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:34.467187 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.467157    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:28:34.471961 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.471936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.474288 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.474267    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2c8631\""
Jul 24 16:28:34.482424 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.482391    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:28:34.515190 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.515156    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:28:34.518610 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.518592    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.521297 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.521262    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2c8631-dockercfg-dftvj\""
Jul 24 16:28:34.529646 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.529619    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:28:34.589738 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.589696    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.589943 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.589882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w7p7\" (UniqueName: \"kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.691076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.690984    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.691076 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.691037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.691309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.691153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w7p7\" (UniqueName: \"kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.691309 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.691217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twvwh\" (UniqueName: \"kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.691748 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.691729    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.704652 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.704624    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w7p7\" (UniqueName: \"kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7\") pod \"payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.784223 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.784168    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:34.791815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.791782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvwh\" (UniqueName: \"kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.792023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.791848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.792674 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.792647    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.802298 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.802271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twvwh\" (UniqueName: \"kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh\") pod \"payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.830920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.830872    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:34.942725 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:34.942402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:28:34.945696 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:28:34.945666    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9749bbc4_dac6_486a_bec1_2e2e5426d049.slice/crio-530b6c5188b5b3a41f70387ea4ca6a748143550ef4980cde4cb3bdaf61ae24b3 WatchSource:0}: Error finding container 530b6c5188b5b3a41f70387ea4ca6a748143550ef4980cde4cb3bdaf61ae24b3: Status 404 returned error can't find the container with id 530b6c5188b5b3a41f70387ea4ca6a748143550ef4980cde4cb3bdaf61ae24b3
Jul 24 16:28:35.010988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.010888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:28:35.013402 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:28:35.013367    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd1765a_07a7_4452_8168_7c4d47e85980.slice/crio-f08c3ebb2498299eeac941ccb03bb4641b8a9432846207cf043938cdc2e62848 WatchSource:0}: Error finding container f08c3ebb2498299eeac941ccb03bb4641b8a9432846207cf043938cdc2e62848: Status 404 returned error can't find the container with id f08c3ebb2498299eeac941ccb03bb4641b8a9432846207cf043938cdc2e62848
Jul 24 16:28:35.103212 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.103175    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" event={"ID":"9749bbc4-dac6-486a-bec1-2e2e5426d049","Type":"ContainerStarted","Data":"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"}
Jul 24 16:28:35.103428 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.103240    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" event={"ID":"9749bbc4-dac6-486a-bec1-2e2e5426d049","Type":"ContainerStarted","Data":"530b6c5188b5b3a41f70387ea4ca6a748143550ef4980cde4cb3bdaf61ae24b3"}
Jul 24 16:28:35.103428 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.103416    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:35.105563 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.105518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" event={"ID":"6c7c834d-1b78-4bcd-a32b-23501fb1d61f","Type":"ContainerStarted","Data":"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"}
Jul 24 16:28:35.105783 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.105576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" event={"ID":"6c7c834d-1b78-4bcd-a32b-23501fb1d61f","Type":"ContainerStarted","Data":"024a3736dbefc1b764027329645154df89da33eb33059ea7e454f37ecc5532c6"}
Jul 24 16:28:35.105783 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.105624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:35.107750 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.107677    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" event={"ID":"cfd1765a-07a7-4452-8168-7c4d47e85980","Type":"ContainerStarted","Data":"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"}
Jul 24 16:28:35.107750 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.107737    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" event={"ID":"cfd1765a-07a7-4452-8168-7c4d47e85980","Type":"ContainerStarted","Data":"f08c3ebb2498299eeac941ccb03bb4641b8a9432846207cf043938cdc2e62848"}
Jul 24 16:28:35.107959 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.107783    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:35.119279 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.119215    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" podStartSLOduration=1.119191394 podStartE2EDuration="1.119191394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:35.118574763 +0000 UTC m=+2856.670814538" watchObservedRunningTime="2026-07-24 16:28:35.119191394 +0000 UTC m=+2856.671431168"
Jul 24 16:28:35.135926 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.135866    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" podStartSLOduration=1.135845936 podStartE2EDuration="1.135845936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:35.133795712 +0000 UTC m=+2856.686035487" watchObservedRunningTime="2026-07-24 16:28:35.135845936 +0000 UTC m=+2856.688085708"
Jul 24 16:28:35.150244 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:35.150183    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" podStartSLOduration=1.600213704 podStartE2EDuration="2.150164037s" totalImagesPullingTime="549.950333ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:28:35.149937594 +0000 UTC m=+2856.702177368" watchObservedRunningTime="2026-07-24 16:28:35.150164037 +0000 UTC m=+2856.702403809"
Jul 24 16:28:37.878538 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:37.878495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:28:41.118888 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:41.118857    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:28:46.115273 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:46.115240    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:28:46.115730 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:46.115320    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:28:55.139495 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:28:55.139395    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd_cfd1765a-07a7-4452-8168-7c4d47e85980/payload-processing/0.log"
Jul 24 16:29:03.413482 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:03.413443    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:06.523648 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:06.523619    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:54.528251 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:54.528220    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd_cfd1765a-07a7-4452-8168-7c4d47e85980/payload-processing/0.log"
Jul 24 16:29:54.811960 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:54.811871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:56.311336 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:56.311302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:57.838170 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.838116    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:57.937187 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.937148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:57.957029 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.956979    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:29:57.962349 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.962317    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:57.965556 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.965526    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-4c4c5c-serving-cert\""
Jul 24 16:29:57.969299 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:57.969247    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:29:58.037471 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.037425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:58.057627 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.057585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hf9\" (UniqueName: \"kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.058023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.057995    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.058257 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.058221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.159611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.159488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hf9\" (UniqueName: \"kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.159611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.159585    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.159611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.159616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.159933 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:29:58.159727    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-serving-cert: secret "maas-api-e2e-ipp-cleanup-4c4c5c-serving-cert" not found
Jul 24 16:29:58.159933 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:29:58.159798    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls podName:2040cafb-c9b3-4d71-8223-bb15caada6d5 nodeName:}" failed. No retries permitted until 2026-07-24 16:29:58.65978055 +0000 UTC m=+2940.212020306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" (UID: "2040cafb-c9b3-4d71-8223-bb15caada6d5") : secret "maas-api-e2e-ipp-cleanup-4c4c5c-serving-cert" not found
Jul 24 16:29:58.160378 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.160351    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.169318 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.169285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hf9\" (UniqueName: \"kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.353468 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.353425    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:58.530938 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.530906    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:58.637276 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.637241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:29:58.656420 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.656386    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:29:58.660212 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.660193    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.662985 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.662963    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4c4c5c-dockercfg-j5z9m\""
Jul 24 16:29:58.663239 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.663225    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4c4c5c\""
Jul 24 16:29:58.665175 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.665146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.668338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.668011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.673357 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.673324    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:29:58.766219 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.766174    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn9mn\" (UniqueName: \"kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.766509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.766314    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.867035 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.866929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn9mn\" (UniqueName: \"kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.867035 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.867007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.869383 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.869359    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-4c4c5c\""
Jul 24 16:29:58.875267 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.875240    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn9mn\" (UniqueName: \"kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.878505 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.878475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:58.879098 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.879080    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:29:58.984714 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.984683    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-4c4c5c-dockercfg-j5z9m\""
Jul 24 16:29:58.992602 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:58.992565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:59.036628 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.036598    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:29:59.038600 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:29:59.038561    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2040cafb_c9b3_4d71_8223_bb15caada6d5.slice/crio-40b683895cf3d622ac79eedd581ba2b3fd19ec43d7adc5a71d7253add05c285e WatchSource:0}: Error finding container 40b683895cf3d622ac79eedd581ba2b3fd19ec43d7adc5a71d7253add05c285e: Status 404 returned error can't find the container with id 40b683895cf3d622ac79eedd581ba2b3fd19ec43d7adc5a71d7253add05c285e
Jul 24 16:29:59.145906 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.145877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:29:59.147909 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:29:59.147876    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3165ce02_518b_4a11_8ce8_cab4f9d219bd.slice/crio-c5bee697fe741dc22da9fb9c7e7bfb9e24fd9f3bb567f8bd23600f8c5b7769f0 WatchSource:0}: Error finding container c5bee697fe741dc22da9fb9c7e7bfb9e24fd9f3bb567f8bd23600f8c5b7769f0: Status 404 returned error can't find the container with id c5bee697fe741dc22da9fb9c7e7bfb9e24fd9f3bb567f8bd23600f8c5b7769f0
Jul 24 16:29:59.527820 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.527780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" event={"ID":"2040cafb-c9b3-4d71-8223-bb15caada6d5","Type":"ContainerStarted","Data":"40b683895cf3d622ac79eedd581ba2b3fd19ec43d7adc5a71d7253add05c285e"}
Jul 24 16:29:59.529491 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.529466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" event={"ID":"3165ce02-518b-4a11-8ce8-cab4f9d219bd","Type":"ContainerStarted","Data":"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"}
Jul 24 16:29:59.529611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.529511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" event={"ID":"3165ce02-518b-4a11-8ce8-cab4f9d219bd","Type":"ContainerStarted","Data":"c5bee697fe741dc22da9fb9c7e7bfb9e24fd9f3bb567f8bd23600f8c5b7769f0"}
Jul 24 16:29:59.529611 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.529562    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:29:59.546181 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:29:59.546108    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" podStartSLOduration=1.54609123 podStartE2EDuration="1.54609123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:29:59.545004617 +0000 UTC m=+2941.097244390" watchObservedRunningTime="2026-07-24 16:29:59.54609123 +0000 UTC m=+2941.098331002"
Jul 24 16:30:00.133013 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.132972    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"]
Jul 24 16:30:00.136652 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.136631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"
Jul 24 16:30:00.139806 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.139779    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:30:00.147761 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.147729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"]
Jul 24 16:30:00.148248 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.148227    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"
Jul 24 16:30:00.301476 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.301447    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"]
Jul 24 16:30:00.303870 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:30:00.303839    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0070ca1c_fe8b_4cb9_95a2_894a0903e099.slice/crio-72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9 WatchSource:0}: Error finding container 72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9: Status 404 returned error can't find the container with id 72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9
Jul 24 16:30:00.536336 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.536297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" event={"ID":"2040cafb-c9b3-4d71-8223-bb15caada6d5","Type":"ContainerStarted","Data":"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"}
Jul 24 16:30:00.536554 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.536455    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:30:00.537624 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.537599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94" event={"ID":"0070ca1c-fe8b-4cb9-95a2-894a0903e099","Type":"ContainerStarted","Data":"72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9"}
Jul 24 16:30:00.555827 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:00.555771    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" podStartSLOduration=2.965866276 podStartE2EDuration="3.555755924s" totalImagesPullingTime="589.889648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:29:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:30:00.55349124 +0000 UTC m=+2942.105731390" watchObservedRunningTime="2026-07-24 16:30:00.555755924 +0000 UTC m=+2942.107995696"
Jul 24 16:30:01.544580 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:01.544543    2567 generic.go:356] "Generic (PLEG): container finished" podID="0070ca1c-fe8b-4cb9-95a2-894a0903e099" containerID="7931e00db31488b5760e942640228ddd30b761be1edd21f40a0f3a4590f76bdc" exitCode=0
Jul 24 16:30:01.544975 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:01.544631    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94" event={"ID":"0070ca1c-fe8b-4cb9-95a2-894a0903e099","Type":"ContainerDied","Data":"7931e00db31488b5760e942640228ddd30b761be1edd21f40a0f3a4590f76bdc"}
Jul 24 16:30:02.334211 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:02.334173    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:02.691014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:02.690983    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"
Jul 24 16:30:03.556349 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:03.556316    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"
Jul 24 16:30:03.556555 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:03.556360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94" event={"ID":"0070ca1c-fe8b-4cb9-95a2-894a0903e099","Type":"ContainerDied","Data":"72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9"}
Jul 24 16:30:03.556555 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:03.556412    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72d74c32eebb7d1f885d2a64fac67f352af98c36569d5edf81fef6195a6905f9"
Jul 24 16:30:06.551026 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:06.550997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:30:10.540204 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:10.540174    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:30:23.748338 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.748238    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:30:23.748774 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.748584    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" podUID="2040cafb-c9b3-4d71-8223-bb15caada6d5" containerName="maas-api" containerID="cri-o://1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f" gracePeriod=30
Jul 24 16:30:23.779313 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.779275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:23.795965 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.795920    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:30:23.796340 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.796285    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" podUID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" containerName="payload-processing" containerID="cri-o://518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d" gracePeriod=30
Jul 24 16:30:23.878450 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.878409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:23.934380 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:23.934330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:24.069434 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.069404    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:30:24.101035 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.101003    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:30:24.188238 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188202    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn9mn\" (UniqueName: \"kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn\") pod \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") "
Jul 24 16:30:24.188467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188264    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82hf9\" (UniqueName: \"kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9\") pod \"2040cafb-c9b3-4d71-8223-bb15caada6d5\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") "
Jul 24 16:30:24.188467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188338    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") pod \"2040cafb-c9b3-4d71-8223-bb15caada6d5\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") "
Jul 24 16:30:24.188467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume\") pod \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\" (UID: \"3165ce02-518b-4a11-8ce8-cab4f9d219bd\") "
Jul 24 16:30:24.188467 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188449    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca\") pod \"2040cafb-c9b3-4d71-8223-bb15caada6d5\" (UID: \"2040cafb-c9b3-4d71-8223-bb15caada6d5\") "
Jul 24 16:30:24.188920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188888    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2040cafb-c9b3-4d71-8223-bb15caada6d5" (UID: "2040cafb-c9b3-4d71-8223-bb15caada6d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:30:24.188920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.188888    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3165ce02-518b-4a11-8ce8-cab4f9d219bd" (UID: "3165ce02-518b-4a11-8ce8-cab4f9d219bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:30:24.190757 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.190727    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2040cafb-c9b3-4d71-8223-bb15caada6d5" (UID: "2040cafb-c9b3-4d71-8223-bb15caada6d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:30:24.190757 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.190734    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9" (OuterVolumeSpecName: "kube-api-access-82hf9") pod "2040cafb-c9b3-4d71-8223-bb15caada6d5" (UID: "2040cafb-c9b3-4d71-8223-bb15caada6d5"). InnerVolumeSpecName "kube-api-access-82hf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:24.190889 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.190744    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn" (OuterVolumeSpecName: "kube-api-access-xn9mn") pod "3165ce02-518b-4a11-8ce8-cab4f9d219bd" (UID: "3165ce02-518b-4a11-8ce8-cab4f9d219bd"). InnerVolumeSpecName "kube-api-access-xn9mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:24.289629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.289540    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2040cafb-c9b3-4d71-8223-bb15caada6d5-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:24.289629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.289574    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3165ce02-518b-4a11-8ce8-cab4f9d219bd-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:24.289629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.289585    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2040cafb-c9b3-4d71-8223-bb15caada6d5-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:24.289629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.289595    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn9mn\" (UniqueName: \"kubernetes.io/projected/3165ce02-518b-4a11-8ce8-cab4f9d219bd-kube-api-access-xn9mn\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:24.289629 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.289605    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82hf9\" (UniqueName: \"kubernetes.io/projected/2040cafb-c9b3-4d71-8223-bb15caada6d5-kube-api-access-82hf9\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:24.672568 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.672471    2567 generic.go:356] "Generic (PLEG): container finished" podID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" containerID="518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d" exitCode=0
Jul 24 16:30:24.672568 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.672521    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"
Jul 24 16:30:24.672568 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.672560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" event={"ID":"3165ce02-518b-4a11-8ce8-cab4f9d219bd","Type":"ContainerDied","Data":"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"}
Jul 24 16:30:24.672875 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.672618    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp" event={"ID":"3165ce02-518b-4a11-8ce8-cab4f9d219bd","Type":"ContainerDied","Data":"c5bee697fe741dc22da9fb9c7e7bfb9e24fd9f3bb567f8bd23600f8c5b7769f0"}
Jul 24 16:30:24.672875 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.672650    2567 scope.go:117] "RemoveContainer" containerID="518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"
Jul 24 16:30:24.674181 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.674154    2567 generic.go:356] "Generic (PLEG): container finished" podID="2040cafb-c9b3-4d71-8223-bb15caada6d5" containerID="1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f" exitCode=0
Jul 24 16:30:24.674341 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.674190    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" event={"ID":"2040cafb-c9b3-4d71-8223-bb15caada6d5","Type":"ContainerDied","Data":"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"}
Jul 24 16:30:24.674341 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.674220    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"
Jul 24 16:30:24.674341 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.674249    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx" event={"ID":"2040cafb-c9b3-4d71-8223-bb15caada6d5","Type":"ContainerDied","Data":"40b683895cf3d622ac79eedd581ba2b3fd19ec43d7adc5a71d7253add05c285e"}
Jul 24 16:30:24.682540 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.682516    2567 scope.go:117] "RemoveContainer" containerID="518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"
Jul 24 16:30:24.682938 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:30:24.682917    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d\": container with ID starting with 518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d not found: ID does not exist" containerID="518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"
Jul 24 16:30:24.683012 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.682952    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d"} err="failed to get container status \"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d\": rpc error: code = NotFound desc = could not find container \"518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d\": container with ID starting with 518882db7e4bd69e62f091c2a22140ad1bd780678025bcf1c5f9779d676cc08d not found: ID does not exist"
Jul 24 16:30:24.683012 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.682981    2567 scope.go:117] "RemoveContainer" containerID="1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"
Jul 24 16:30:24.692402 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.692376    2567 scope.go:117] "RemoveContainer" containerID="1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"
Jul 24 16:30:24.692726 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:30:24.692699    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f\": container with ID starting with 1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f not found: ID does not exist" containerID="1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"
Jul 24 16:30:24.692799 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.692736    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f"} err="failed to get container status \"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f\": rpc error: code = NotFound desc = could not find container \"1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f\": container with ID starting with 1d0c2d3176acd26316b8e95aa63587ed8e64bfe86312560b30f24d3d34a9110f not found: ID does not exist"
Jul 24 16:30:24.697444 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.697416    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:30:24.701278 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.701252    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-4c4c5c-7fbd9df576-p7fhp"]
Jul 24 16:30:24.711643 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.711609    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:30:24.715231 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.715204    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-4c4c5c-67d48ff664-956zx"]
Jul 24 16:30:24.949519 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.949430    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2040cafb-c9b3-4d71-8223-bb15caada6d5" path="/var/lib/kubelet/pods/2040cafb-c9b3-4d71-8223-bb15caada6d5/volumes"
Jul 24 16:30:24.949895 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:24.949784    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" path="/var/lib/kubelet/pods/3165ce02-518b-4a11-8ce8-cab4f9d219bd/volumes"
Jul 24 16:30:28.366609 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:28.366577    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:28.761735 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:28.761702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:34.380508 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:34.380455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:34.435460 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:34.435421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:34.659700 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:34.659606    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:34.895861 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:34.895827    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:53.854678 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.854636    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:30:53.855182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.854871    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" podUID="27344a50-35c4-4893-b4d9-7f64d64b296d" containerName="maas-api" containerID="cri-o://9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385" gracePeriod=30
Jul 24 16:30:53.885821 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.885786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:53.916490 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.916451    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:30:53.916759 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.916714    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" podUID="47071e4a-ff13-4795-90a3-4af33734328f" containerName="payload-pre-processing" containerID="cri-o://1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53" gracePeriod=30
Jul 24 16:30:53.983925 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:53.983888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:54.087169 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.087033    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:54.182531 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.182500    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:30:54.215630 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.215587    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:30:54.360227 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360190    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxrb6\" (UniqueName: \"kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6\") pod \"27344a50-35c4-4893-b4d9-7f64d64b296d\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") "
Jul 24 16:30:54.360414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360266    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca\") pod \"27344a50-35c4-4893-b4d9-7f64d64b296d\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") "
Jul 24 16:30:54.360414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360288    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume\") pod \"47071e4a-ff13-4795-90a3-4af33734328f\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") "
Jul 24 16:30:54.360414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360320    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") pod \"27344a50-35c4-4893-b4d9-7f64d64b296d\" (UID: \"27344a50-35c4-4893-b4d9-7f64d64b296d\") "
Jul 24 16:30:54.360414 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360361    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prf4q\" (UniqueName: \"kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q\") pod \"47071e4a-ff13-4795-90a3-4af33734328f\" (UID: \"47071e4a-ff13-4795-90a3-4af33734328f\") "
Jul 24 16:30:54.360691 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360669    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27344a50-35c4-4893-b4d9-7f64d64b296d" (UID: "27344a50-35c4-4893-b4d9-7f64d64b296d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:30:54.360755 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.360716    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47071e4a-ff13-4795-90a3-4af33734328f" (UID: "47071e4a-ff13-4795-90a3-4af33734328f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:30:54.362675 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.362638    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q" (OuterVolumeSpecName: "kube-api-access-prf4q") pod "47071e4a-ff13-4795-90a3-4af33734328f" (UID: "47071e4a-ff13-4795-90a3-4af33734328f"). InnerVolumeSpecName "kube-api-access-prf4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:54.362786 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.362694    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27344a50-35c4-4893-b4d9-7f64d64b296d" (UID: "27344a50-35c4-4893-b4d9-7f64d64b296d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:30:54.362786 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.362762    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6" (OuterVolumeSpecName: "kube-api-access-xxrb6") pod "27344a50-35c4-4893-b4d9-7f64d64b296d" (UID: "27344a50-35c4-4893-b4d9-7f64d64b296d"). InnerVolumeSpecName "kube-api-access-xxrb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:54.462101 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.461994    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27344a50-35c4-4893-b4d9-7f64d64b296d-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:54.462101 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.462033    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47071e4a-ff13-4795-90a3-4af33734328f-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:54.462101 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.462044    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27344a50-35c4-4893-b4d9-7f64d64b296d-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:54.462101 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.462053    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prf4q\" (UniqueName: \"kubernetes.io/projected/47071e4a-ff13-4795-90a3-4af33734328f-kube-api-access-prf4q\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:54.462101 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.462062    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrb6\" (UniqueName: \"kubernetes.io/projected/27344a50-35c4-4893-b4d9-7f64d64b296d-kube-api-access-xxrb6\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:30:54.827880 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.827847    2567 generic.go:356] "Generic (PLEG): container finished" podID="47071e4a-ff13-4795-90a3-4af33734328f" containerID="1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53" exitCode=0
Jul 24 16:30:54.828159 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.827889    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"
Jul 24 16:30:54.828159 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.827938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" event={"ID":"47071e4a-ff13-4795-90a3-4af33734328f","Type":"ContainerDied","Data":"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"}
Jul 24 16:30:54.828159 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.827992    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc" event={"ID":"47071e4a-ff13-4795-90a3-4af33734328f","Type":"ContainerDied","Data":"26ccb7048143c80b1ce0e860cf4ba87a89d7e465edea19ba44ddcff64ae7bb43"}
Jul 24 16:30:54.828159 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.828015    2567 scope.go:117] "RemoveContainer" containerID="1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"
Jul 24 16:30:54.829525 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.829501    2567 generic.go:356] "Generic (PLEG): container finished" podID="27344a50-35c4-4893-b4d9-7f64d64b296d" containerID="9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385" exitCode=0
Jul 24 16:30:54.829624 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.829569    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" event={"ID":"27344a50-35c4-4893-b4d9-7f64d64b296d","Type":"ContainerDied","Data":"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"}
Jul 24 16:30:54.829624 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.829572    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"
Jul 24 16:30:54.829624 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.829604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl" event={"ID":"27344a50-35c4-4893-b4d9-7f64d64b296d","Type":"ContainerDied","Data":"c24d5900e25cf1bada3e027cf5400410ec34d2f083daf75828fbfe217d028543"}
Jul 24 16:30:54.846800 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.846770    2567 scope.go:117] "RemoveContainer" containerID="1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"
Jul 24 16:30:54.847163 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:30:54.847112    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53\": container with ID starting with 1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53 not found: ID does not exist" containerID="1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"
Jul 24 16:30:54.847216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.847174    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53"} err="failed to get container status \"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53\": rpc error: code = NotFound desc = could not find container \"1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53\": container with ID starting with 1b5a903fdac918d6b77f9c454d7164a3085080cce461a006756c1b64f48a3a53 not found: ID does not exist"
Jul 24 16:30:54.847216 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.847193    2567 scope.go:117] "RemoveContainer" containerID="9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"
Jul 24 16:30:54.857014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.856972    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:30:54.857470 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.857442    2567 scope.go:117] "RemoveContainer" containerID="9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"
Jul 24 16:30:54.857842 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:30:54.857822    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385\": container with ID starting with 9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385 not found: ID does not exist" containerID="9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"
Jul 24 16:30:54.857889 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.857851    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385"} err="failed to get container status \"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385\": rpc error: code = NotFound desc = could not find container \"9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385\": container with ID starting with 9337a3f80435519fc2724c5d445cd9f5473c4cfddfe3e98e3c63c12d96b35385 not found: ID does not exist"
Jul 24 16:30:54.861258 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.861227    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2c5dfe-78db64d676-82qrc"]
Jul 24 16:30:54.872192 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.872162    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:30:54.875614 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.875586    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2c5dfe-6c944c6b78-rrbxl"]
Jul 24 16:30:54.949742 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.949707    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27344a50-35c4-4893-b4d9-7f64d64b296d" path="/var/lib/kubelet/pods/27344a50-35c4-4893-b4d9-7f64d64b296d/volumes"
Jul 24 16:30:54.950086 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:54.950072    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47071e4a-ff13-4795-90a3-4af33734328f" path="/var/lib/kubelet/pods/47071e4a-ff13-4795-90a3-4af33734328f/volumes"
Jul 24 16:30:58.390370 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:58.390331    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:30:58.776530 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:30:58.776497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:03.798920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:03.798881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:17.235176 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:17.235116    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:17.287061 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:17.287025    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:40.403082 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:40.403046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:40.831176 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:40.831087    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:46.228882 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:46.228844    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:46.339289 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:46.339250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:46.538116 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:46.538085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:46.792795 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:46.792710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:57.681422 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.681194    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:31:57.681867 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.681579    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" podUID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" containerName="maas-api" containerID="cri-o://197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711" gracePeriod=30
Jul 24 16:31:57.718051 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.718020    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:57.743399 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.743365    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:31:57.744355 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.744290    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" podUID="cfd1765a-07a7-4452-8168-7c4d47e85980" containerName="payload-processing" containerID="cri-o://8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb" gracePeriod=30
Jul 24 16:31:57.749250 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.749221    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:31:57.751289 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.751253    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" podUID="9749bbc4-dac6-486a-bec1-2e2e5426d049" containerName="payload-pre-processing" containerID="cri-o://596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb" gracePeriod=30
Jul 24 16:31:57.784856 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.784825    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:57.885585 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:57.885539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:31:58.022461 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.022432    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:31:58.038920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.038589    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4cqn\" (UniqueName: \"kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn\") pod \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") "
Jul 24 16:31:58.038920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.038682    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls\") pod \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") "
Jul 24 16:31:58.038920 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.038781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca\") pod \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\" (UID: \"6c7c834d-1b78-4bcd-a32b-23501fb1d61f\") "
Jul 24 16:31:58.039334 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.039313    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6c7c834d-1b78-4bcd-a32b-23501fb1d61f" (UID: "6c7c834d-1b78-4bcd-a32b-23501fb1d61f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:31:58.042477 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.042431    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn" (OuterVolumeSpecName: "kube-api-access-t4cqn") pod "6c7c834d-1b78-4bcd-a32b-23501fb1d61f" (UID: "6c7c834d-1b78-4bcd-a32b-23501fb1d61f"). InnerVolumeSpecName "kube-api-access-t4cqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:31:58.043804 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.043775    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6c7c834d-1b78-4bcd-a32b-23501fb1d61f" (UID: "6c7c834d-1b78-4bcd-a32b-23501fb1d61f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:31:58.062242 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.062211    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:31:58.074024 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.074000    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:31:58.139574 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.139534    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twvwh\" (UniqueName: \"kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh\") pod \"cfd1765a-07a7-4452-8168-7c4d47e85980\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") "
Jul 24 16:31:58.139770 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.139608    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume\") pod \"9749bbc4-dac6-486a-bec1-2e2e5426d049\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") "
Jul 24 16:31:58.139770 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.139639    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume\") pod \"cfd1765a-07a7-4452-8168-7c4d47e85980\" (UID: \"cfd1765a-07a7-4452-8168-7c4d47e85980\") "
Jul 24 16:31:58.139770 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.139744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w7p7\" (UniqueName: \"kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7\") pod \"9749bbc4-dac6-486a-bec1-2e2e5426d049\" (UID: \"9749bbc4-dac6-486a-bec1-2e2e5426d049\") "
Jul 24 16:31:58.140038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.140008    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cfd1765a-07a7-4452-8168-7c4d47e85980" (UID: "cfd1765a-07a7-4452-8168-7c4d47e85980"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:31:58.140038 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.140012    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9749bbc4-dac6-486a-bec1-2e2e5426d049" (UID: "9749bbc4-dac6-486a-bec1-2e2e5426d049"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:31:58.140229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.140084    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.140229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.140106    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4cqn\" (UniqueName: \"kubernetes.io/projected/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-kube-api-access-t4cqn\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.140229 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.140121    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6c7c834d-1b78-4bcd-a32b-23501fb1d61f-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.141958 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.141929    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh" (OuterVolumeSpecName: "kube-api-access-twvwh") pod "cfd1765a-07a7-4452-8168-7c4d47e85980" (UID: "cfd1765a-07a7-4452-8168-7c4d47e85980"). InnerVolumeSpecName "kube-api-access-twvwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:31:58.142049 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.142003    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7" (OuterVolumeSpecName: "kube-api-access-7w7p7") pod "9749bbc4-dac6-486a-bec1-2e2e5426d049" (UID: "9749bbc4-dac6-486a-bec1-2e2e5426d049"). InnerVolumeSpecName "kube-api-access-7w7p7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:31:58.157846 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.157819    2567 generic.go:356] "Generic (PLEG): container finished" podID="9749bbc4-dac6-486a-bec1-2e2e5426d049" containerID="596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb" exitCode=0
Jul 24 16:31:58.157997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.157867    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"
Jul 24 16:31:58.157997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.157901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" event={"ID":"9749bbc4-dac6-486a-bec1-2e2e5426d049","Type":"ContainerDied","Data":"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"}
Jul 24 16:31:58.157997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.157950    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls" event={"ID":"9749bbc4-dac6-486a-bec1-2e2e5426d049","Type":"ContainerDied","Data":"530b6c5188b5b3a41f70387ea4ca6a748143550ef4980cde4cb3bdaf61ae24b3"}
Jul 24 16:31:58.157997 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.157973    2567 scope.go:117] "RemoveContainer" containerID="596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"
Jul 24 16:31:58.159411 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.159394    2567 generic.go:356] "Generic (PLEG): container finished" podID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" containerID="197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711" exitCode=0
Jul 24 16:31:58.159510 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.159450    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"
Jul 24 16:31:58.159582 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.159494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" event={"ID":"6c7c834d-1b78-4bcd-a32b-23501fb1d61f","Type":"ContainerDied","Data":"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"}
Jul 24 16:31:58.160366 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.159624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw" event={"ID":"6c7c834d-1b78-4bcd-a32b-23501fb1d61f","Type":"ContainerDied","Data":"024a3736dbefc1b764027329645154df89da33eb33059ea7e454f37ecc5532c6"}
Jul 24 16:31:58.160963 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.160944    2567 generic.go:356] "Generic (PLEG): container finished" podID="cfd1765a-07a7-4452-8168-7c4d47e85980" containerID="8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb" exitCode=0
Jul 24 16:31:58.161077 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.160975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" event={"ID":"cfd1765a-07a7-4452-8168-7c4d47e85980","Type":"ContainerDied","Data":"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"}
Jul 24 16:31:58.161077 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.160993    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"
Jul 24 16:31:58.161077 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.161010    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd" event={"ID":"cfd1765a-07a7-4452-8168-7c4d47e85980","Type":"ContainerDied","Data":"f08c3ebb2498299eeac941ccb03bb4641b8a9432846207cf043938cdc2e62848"}
Jul 24 16:31:58.169612 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.169549    2567 scope.go:117] "RemoveContainer" containerID="596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"
Jul 24 16:31:58.169877 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:31:58.169855    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb\": container with ID starting with 596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb not found: ID does not exist" containerID="596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"
Jul 24 16:31:58.169982 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.169883    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb"} err="failed to get container status \"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb\": rpc error: code = NotFound desc = could not find container \"596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb\": container with ID starting with 596522ee89e5a2f3b3fc16c7b629edee30648ac84c9db9bb2fe59a72f96860cb not found: ID does not exist"
Jul 24 16:31:58.169982 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.169900    2567 scope.go:117] "RemoveContainer" containerID="197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"
Jul 24 16:31:58.178618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.178528    2567 scope.go:117] "RemoveContainer" containerID="197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"
Jul 24 16:31:58.178928 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:31:58.178905    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711\": container with ID starting with 197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711 not found: ID does not exist" containerID="197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"
Jul 24 16:31:58.179014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.178936    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711"} err="failed to get container status \"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711\": rpc error: code = NotFound desc = could not find container \"197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711\": container with ID starting with 197eef3ce943bff6c2abfdfe2ec36f7201a017c95aa39730e643e09cab9f0711 not found: ID does not exist"
Jul 24 16:31:58.179014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.178953    2567 scope.go:117] "RemoveContainer" containerID="8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"
Jul 24 16:31:58.180519 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.180498    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:31:58.183621 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.183599    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2c8631-5d7c8cc956-9htls"]
Jul 24 16:31:58.187663 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.187646    2567 scope.go:117] "RemoveContainer" containerID="8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"
Jul 24 16:31:58.187914 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:31:58.187897    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb\": container with ID starting with 8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb not found: ID does not exist" containerID="8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"
Jul 24 16:31:58.187953 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.187922    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb"} err="failed to get container status \"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb\": rpc error: code = NotFound desc = could not find container \"8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb\": container with ID starting with 8a923d558ee17a97bf62498497120626c8ceca2a036cb479fee32465621343cb not found: ID does not exist"
Jul 24 16:31:58.194277 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.194256    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:31:58.195914 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.195892    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2c8631-7459d8b678-b9bpd"]
Jul 24 16:31:58.205675 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.205653    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:31:58.207850 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.207829    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2c8631-ff977f6ff-4zxbw"]
Jul 24 16:31:58.241509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.241471    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9749bbc4-dac6-486a-bec1-2e2e5426d049-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.241509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.241504    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfd1765a-07a7-4452-8168-7c4d47e85980-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.241509 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.241515    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w7p7\" (UniqueName: \"kubernetes.io/projected/9749bbc4-dac6-486a-bec1-2e2e5426d049-kube-api-access-7w7p7\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.241710 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.241525    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twvwh\" (UniqueName: \"kubernetes.io/projected/cfd1765a-07a7-4452-8168-7c4d47e85980-kube-api-access-twvwh\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:31:58.950426 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.950335    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" path="/var/lib/kubelet/pods/6c7c834d-1b78-4bcd-a32b-23501fb1d61f/volumes"
Jul 24 16:31:58.950890 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.950870    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9749bbc4-dac6-486a-bec1-2e2e5426d049" path="/var/lib/kubelet/pods/9749bbc4-dac6-486a-bec1-2e2e5426d049/volumes"
Jul 24 16:31:58.951464 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:31:58.951332    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd1765a-07a7-4452-8168-7c4d47e85980" path="/var/lib/kubelet/pods/cfd1765a-07a7-4452-8168-7c4d47e85980/volumes"
Jul 24 16:32:02.001066 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:02.001029    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:02.374570 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:02.374482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:08.155787 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:08.155755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:08.258023 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:08.257988    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:08.428424 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:08.428335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:08.667842 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:08.667806    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:19.231851 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.231549    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:32:19.232532 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.232465    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" podUID="6e2f08a9-259d-4fab-85be-d5043c97602e" containerName="maas-api" containerID="cri-o://bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca" gracePeriod=30
Jul 24 16:32:19.256138 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.256091    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"]
Jul 24 16:32:19.258433 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.258403    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-ed0643-29748510-mvx94"]
Jul 24 16:32:19.260014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.259984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:19.286408 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.286373    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:32:19.286618 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.286591    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" podUID="20f7a349-7926-4781-ad00-bd9bb04f4d25" containerName="payload-processing" containerID="cri-o://f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1" gracePeriod=30
Jul 24 16:32:19.296115 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.296080    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:32:19.296333 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.296310    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" podUID="7db665e4-83c7-4a69-b688-4af5a1b66996" containerName="payload-pre-processing" containerID="cri-o://5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8" gracePeriod=30
Jul 24 16:32:19.325063 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.325034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:19.425313 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.425275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:19.579806 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.579775    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:32:19.616455 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.616418    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:32:19.628535 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.628504    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:32:19.635877 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.635845    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") pod \"6e2f08a9-259d-4fab-85be-d5043c97602e\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") "
Jul 24 16:32:19.635877 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.635890    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdp7f\" (UniqueName: \"kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f\") pod \"6e2f08a9-259d-4fab-85be-d5043c97602e\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") "
Jul 24 16:32:19.636192 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.636153    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca\") pod \"6e2f08a9-259d-4fab-85be-d5043c97602e\" (UID: \"6e2f08a9-259d-4fab-85be-d5043c97602e\") "
Jul 24 16:32:19.636839 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.636806    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e2f08a9-259d-4fab-85be-d5043c97602e" (UID: "6e2f08a9-259d-4fab-85be-d5043c97602e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:19.638540 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.638509    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e2f08a9-259d-4fab-85be-d5043c97602e" (UID: "6e2f08a9-259d-4fab-85be-d5043c97602e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:32:19.638657 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.638611    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f" (OuterVolumeSpecName: "kube-api-access-sdp7f") pod "6e2f08a9-259d-4fab-85be-d5043c97602e" (UID: "6e2f08a9-259d-4fab-85be-d5043c97602e"). InnerVolumeSpecName "kube-api-access-sdp7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:19.737659 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume\") pod \"7db665e4-83c7-4a69-b688-4af5a1b66996\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") "
Jul 24 16:32:19.737825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737723    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djbv4\" (UniqueName: \"kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4\") pod \"7db665e4-83c7-4a69-b688-4af5a1b66996\" (UID: \"7db665e4-83c7-4a69-b688-4af5a1b66996\") "
Jul 24 16:32:19.737825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkblb\" (UniqueName: \"kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb\") pod \"20f7a349-7926-4781-ad00-bd9bb04f4d25\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") "
Jul 24 16:32:19.737825 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737779    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume\") pod \"20f7a349-7926-4781-ad00-bd9bb04f4d25\" (UID: \"20f7a349-7926-4781-ad00-bd9bb04f4d25\") "
Jul 24 16:32:19.738020 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737970    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e2f08a9-259d-4fab-85be-d5043c97602e-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.738020 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.737989    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdp7f\" (UniqueName: \"kubernetes.io/projected/6e2f08a9-259d-4fab-85be-d5043c97602e-kube-api-access-sdp7f\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.738020 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.738003    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e2f08a9-259d-4fab-85be-d5043c97602e-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.738220 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.738162    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7db665e4-83c7-4a69-b688-4af5a1b66996" (UID: "7db665e4-83c7-4a69-b688-4af5a1b66996"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:19.738279 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.738257    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "20f7a349-7926-4781-ad00-bd9bb04f4d25" (UID: "20f7a349-7926-4781-ad00-bd9bb04f4d25"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:19.740177 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.740152    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4" (OuterVolumeSpecName: "kube-api-access-djbv4") pod "7db665e4-83c7-4a69-b688-4af5a1b66996" (UID: "7db665e4-83c7-4a69-b688-4af5a1b66996"). InnerVolumeSpecName "kube-api-access-djbv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:19.740177 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.740161    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb" (OuterVolumeSpecName: "kube-api-access-zkblb") pod "20f7a349-7926-4781-ad00-bd9bb04f4d25" (UID: "20f7a349-7926-4781-ad00-bd9bb04f4d25"). InnerVolumeSpecName "kube-api-access-zkblb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:19.839102 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.838993    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djbv4\" (UniqueName: \"kubernetes.io/projected/7db665e4-83c7-4a69-b688-4af5a1b66996-kube-api-access-djbv4\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.839102 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.839029    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkblb\" (UniqueName: \"kubernetes.io/projected/20f7a349-7926-4781-ad00-bd9bb04f4d25-kube-api-access-zkblb\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.839102 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.839042    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f7a349-7926-4781-ad00-bd9bb04f4d25-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:19.839102 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:19.839057    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7db665e4-83c7-4a69-b688-4af5a1b66996-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:20.271466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.271426    2567 generic.go:356] "Generic (PLEG): container finished" podID="6e2f08a9-259d-4fab-85be-d5043c97602e" containerID="bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca" exitCode=0
Jul 24 16:32:20.271466 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.271469    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"
Jul 24 16:32:20.271994 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.271518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" event={"ID":"6e2f08a9-259d-4fab-85be-d5043c97602e","Type":"ContainerDied","Data":"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"}
Jul 24 16:32:20.271994 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.271567    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k" event={"ID":"6e2f08a9-259d-4fab-85be-d5043c97602e","Type":"ContainerDied","Data":"a1e8640bf83ca18f8f7efca8b623730091126e531a9ecf1bf5c5e28bbb2d12c9"}
Jul 24 16:32:20.271994 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.271589    2567 scope.go:117] "RemoveContainer" containerID="bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"
Jul 24 16:32:20.273091 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.272927    2567 generic.go:356] "Generic (PLEG): container finished" podID="20f7a349-7926-4781-ad00-bd9bb04f4d25" containerID="f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1" exitCode=0
Jul 24 16:32:20.273091 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.273073    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" event={"ID":"20f7a349-7926-4781-ad00-bd9bb04f4d25","Type":"ContainerDied","Data":"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"}
Jul 24 16:32:20.273344 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.273185    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg" event={"ID":"20f7a349-7926-4781-ad00-bd9bb04f4d25","Type":"ContainerDied","Data":"b46fd8a8bbfcae2b7ae44866bf518454cf2353d36dfd7b9d52cc5c89a63c4bee"}
Jul 24 16:32:20.273512 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.273492    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"
Jul 24 16:32:20.274535 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.274514    2567 generic.go:356] "Generic (PLEG): container finished" podID="7db665e4-83c7-4a69-b688-4af5a1b66996" containerID="5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8" exitCode=0
Jul 24 16:32:20.274661 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.274564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"
Jul 24 16:32:20.274726 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.274569    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" event={"ID":"7db665e4-83c7-4a69-b688-4af5a1b66996","Type":"ContainerDied","Data":"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"}
Jul 24 16:32:20.274781 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.274742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz" event={"ID":"7db665e4-83c7-4a69-b688-4af5a1b66996","Type":"ContainerDied","Data":"21f3ee4e1ddcb6e16df329702664521d87354026eacb7c5ea301691176088a2b"}
Jul 24 16:32:20.285681 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.285654    2567 scope.go:117] "RemoveContainer" containerID="bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"
Jul 24 16:32:20.286096 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:32:20.286059    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca\": container with ID starting with bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca not found: ID does not exist" containerID="bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"
Jul 24 16:32:20.286235 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.286111    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca"} err="failed to get container status \"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca\": rpc error: code = NotFound desc = could not find container \"bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca\": container with ID starting with bc6422d5ca3f70ab371f0fd754521cadd325cd3928d97a7c3be22d54a3200eca not found: ID does not exist"
Jul 24 16:32:20.286235 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.286157    2567 scope.go:117] "RemoveContainer" containerID="f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"
Jul 24 16:32:20.295296 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.295268    2567 scope.go:117] "RemoveContainer" containerID="f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"
Jul 24 16:32:20.295703 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:32:20.295680    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1\": container with ID starting with f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1 not found: ID does not exist" containerID="f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"
Jul 24 16:32:20.295810 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.295714    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1"} err="failed to get container status \"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1\": rpc error: code = NotFound desc = could not find container \"f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1\": container with ID starting with f9777a02e0550b584c40f09dc5705c70278999e017a957f01abcd1a1d421f8f1 not found: ID does not exist"
Jul 24 16:32:20.295810 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.295741    2567 scope.go:117] "RemoveContainer" containerID="5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"
Jul 24 16:32:20.297398 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.297374    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:32:20.301420 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.301396    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-ed0643-cc77b4547-vcw2k"]
Jul 24 16:32:20.305752 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.305726    2567 scope.go:117] "RemoveContainer" containerID="5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"
Jul 24 16:32:20.306236 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:32:20.306203    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8\": container with ID starting with 5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8 not found: ID does not exist" containerID="5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"
Jul 24 16:32:20.306340 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.306249    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8"} err="failed to get container status \"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8\": rpc error: code = NotFound desc = could not find container \"5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8\": container with ID starting with 5e12f6bda972f0f97b8ad324f55bd604434d000520faf2aae3c020a446a805f8 not found: ID does not exist"
Jul 24 16:32:20.316815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.316751    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:32:20.318673 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.318644    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-ed0643-7959f488d6-mzvsg"]
Jul 24 16:32:20.329176 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.329141    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:32:20.332935 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.332898    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-ed0643-5f86fb7cdd-2xjkz"]
Jul 24 16:32:20.949146 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.949071    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0070ca1c-fe8b-4cb9-95a2-894a0903e099" path="/var/lib/kubelet/pods/0070ca1c-fe8b-4cb9-95a2-894a0903e099/volumes"
Jul 24 16:32:20.949527 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.949507    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f7a349-7926-4781-ad00-bd9bb04f4d25" path="/var/lib/kubelet/pods/20f7a349-7926-4781-ad00-bd9bb04f4d25/volumes"
Jul 24 16:32:20.949930 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.949909    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e2f08a9-259d-4fab-85be-d5043c97602e" path="/var/lib/kubelet/pods/6e2f08a9-259d-4fab-85be-d5043c97602e/volumes"
Jul 24 16:32:20.950313 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:20.950295    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7db665e4-83c7-4a69-b688-4af5a1b66996" path="/var/lib/kubelet/pods/7db665e4-83c7-4a69-b688-4af5a1b66996/volumes"
Jul 24 16:32:23.884214 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:23.884179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:24.304471 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:24.304438    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:30.174313 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.174278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:30.275884 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.275856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:30.468249 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.468109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:30.724195 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.724088    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:30.851304 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851267    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"]
Jul 24 16:32:30.851768 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851752    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7db665e4-83c7-4a69-b688-4af5a1b66996" containerName="payload-pre-processing"
Jul 24 16:32:30.851815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851773    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7db665e4-83c7-4a69-b688-4af5a1b66996" containerName="payload-pre-processing"
Jul 24 16:32:30.851815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851791    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd1765a-07a7-4452-8168-7c4d47e85980" containerName="payload-processing"
Jul 24 16:32:30.851815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851800    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd1765a-07a7-4452-8168-7c4d47e85980" containerName="payload-processing"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851822    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e2f08a9-259d-4fab-85be-d5043c97602e" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851831    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e2f08a9-259d-4fab-85be-d5043c97602e" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851844    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27344a50-35c4-4893-b4d9-7f64d64b296d" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851853    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27344a50-35c4-4893-b4d9-7f64d64b296d" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851868    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2040cafb-c9b3-4d71-8223-bb15caada6d5" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851877    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2040cafb-c9b3-4d71-8223-bb15caada6d5" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851888    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851896    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" containerName="maas-api"
Jul 24 16:32:30.851909 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851905    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9749bbc4-dac6-486a-bec1-2e2e5426d049" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851914    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9749bbc4-dac6-486a-bec1-2e2e5426d049" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851923    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0070ca1c-fe8b-4cb9-95a2-894a0903e099" containerName="cleanup"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851933    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0070ca1c-fe8b-4cb9-95a2-894a0903e099" containerName="cleanup"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851944    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851952    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851962    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f7a349-7926-4781-ad00-bd9bb04f4d25" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851970    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f7a349-7926-4781-ad00-bd9bb04f4d25" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851980    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47071e4a-ff13-4795-90a3-4af33734328f" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.851988    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="47071e4a-ff13-4795-90a3-4af33734328f" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852064    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3165ce02-518b-4a11-8ce8-cab4f9d219bd" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852077    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2040cafb-c9b3-4d71-8223-bb15caada6d5" containerName="maas-api"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852086    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27344a50-35c4-4893-b4d9-7f64d64b296d" containerName="maas-api"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852094    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c7c834d-1b78-4bcd-a32b-23501fb1d61f" containerName="maas-api"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852103    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7db665e4-83c7-4a69-b688-4af5a1b66996" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852111    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f7a349-7926-4781-ad00-bd9bb04f4d25" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852119    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd1765a-07a7-4452-8168-7c4d47e85980" containerName="payload-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852148    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0070ca1c-fe8b-4cb9-95a2-894a0903e099" containerName="cleanup"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852155    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e2f08a9-259d-4fab-85be-d5043c97602e" containerName="maas-api"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852162    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9749bbc4-dac6-486a-bec1-2e2e5426d049" containerName="payload-pre-processing"
Jul 24 16:32:30.852196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.852169    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="47071e4a-ff13-4795-90a3-4af33734328f" containerName="payload-pre-processing"
Jul 24 16:32:30.857090 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.857071    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:30.859842 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.859817    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b924x\""
Jul 24 16:32:30.861871 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.861845    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"]
Jul 24 16:32:30.942249 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:30.942205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw\" (UID: \"7b24deec-4134-452e-b1a4-5815f9ada508\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:31.043177 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:31.043112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw\" (UID: \"7b24deec-4134-452e-b1a4-5815f9ada508\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:31.043721 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:31.043699    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw\" (UID: \"7b24deec-4134-452e-b1a4-5815f9ada508\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:31.170257 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:31.170216    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:31.313988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:31.313871    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"]
Jul 24 16:32:31.316617 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:32:31.316587    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b24deec_4134_452e_b1a4_5815f9ada508.slice/crio-da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300 WatchSource:0}: Error finding container da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300: Status 404 returned error can't find the container with id da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300
Jul 24 16:32:31.327895 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:31.327869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw" event={"ID":"7b24deec-4134-452e-b1a4-5815f9ada508","Type":"ContainerStarted","Data":"da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300"}
Jul 24 16:32:32.334205 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:32.334163    2567 generic.go:356] "Generic (PLEG): container finished" podID="7b24deec-4134-452e-b1a4-5815f9ada508" containerID="9d54597733de2bfd0f45b616c8979bb985573cc3cc96f62f59d5c677d80c1d03" exitCode=0
Jul 24 16:32:32.334641 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:32.334308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw" event={"ID":"7b24deec-4134-452e-b1a4-5815f9ada508","Type":"ContainerDied","Data":"9d54597733de2bfd0f45b616c8979bb985573cc3cc96f62f59d5c677d80c1d03"}
Jul 24 16:32:33.470948 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:33.470923    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:33.565941 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:33.565901    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca\") pod \"7b24deec-4134-452e-b1a4-5815f9ada508\" (UID: \"7b24deec-4134-452e-b1a4-5815f9ada508\") "
Jul 24 16:32:33.566346 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:33.566320    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b24deec-4134-452e-b1a4-5815f9ada508" (UID: "7b24deec-4134-452e-b1a4-5815f9ada508"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:33.666880 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:33.666783    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b24deec-4134-452e-b1a4-5815f9ada508-maas-api-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:34.345744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:34.345707    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"
Jul 24 16:32:34.345744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:34.345733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw" event={"ID":"7b24deec-4134-452e-b1a4-5815f9ada508","Type":"ContainerDied","Data":"da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300"}
Jul 24 16:32:34.345962 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:34.345780    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da2193c7263f23639ad3c68634027b46f49056e5317af8bb766f9f5eecb8d300"
Jul 24 16:32:41.374862 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.374820    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:32:41.375402 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.375102    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" podUID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" containerName="maas-api" containerID="cri-o://d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a" gracePeriod=30
Jul 24 16:32:41.408173 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.408143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:41.437859 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.437818    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:32:41.438311 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.438103    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" podUID="75ebcf41-28fa-4df5-9636-c5c129209c46" containerName="payload-pre-processing" containerID="cri-o://17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5" gracePeriod=30
Jul 24 16:32:41.507963 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.507933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:41.611713 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.611548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:41.695411 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.695388    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:32:41.722528 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.722500    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:32:41.733359 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733334    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd6b7\" (UniqueName: \"kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7\") pod \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") "
Jul 24 16:32:41.733359 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733364    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpfwd\" (UniqueName: \"kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd\") pod \"75ebcf41-28fa-4df5-9636-c5c129209c46\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") "
Jul 24 16:32:41.733556 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733405    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume\") pod \"75ebcf41-28fa-4df5-9636-c5c129209c46\" (UID: \"75ebcf41-28fa-4df5-9636-c5c129209c46\") "
Jul 24 16:32:41.733556 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733446    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca\") pod \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") "
Jul 24 16:32:41.733665 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733602    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") pod \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\" (UID: \"c232c7e4-6193-48d3-8bd3-c978a9a3d73d\") "
Jul 24 16:32:41.733837 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733813    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c232c7e4-6193-48d3-8bd3-c978a9a3d73d" (UID: "c232c7e4-6193-48d3-8bd3-c978a9a3d73d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:41.733976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733841    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75ebcf41-28fa-4df5-9636-c5c129209c46" (UID: "75ebcf41-28fa-4df5-9636-c5c129209c46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:32:41.733976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733932    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75ebcf41-28fa-4df5-9636-c5c129209c46-plugins-config-volume\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:41.733976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.733947    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-openshift-service-ca\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:41.735823 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.735794    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c232c7e4-6193-48d3-8bd3-c978a9a3d73d" (UID: "c232c7e4-6193-48d3-8bd3-c978a9a3d73d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:32:41.735823 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.735802    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd" (OuterVolumeSpecName: "kube-api-access-dpfwd") pod "75ebcf41-28fa-4df5-9636-c5c129209c46" (UID: "75ebcf41-28fa-4df5-9636-c5c129209c46"). InnerVolumeSpecName "kube-api-access-dpfwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:41.735953 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.735822    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7" (OuterVolumeSpecName: "kube-api-access-xd6b7") pod "c232c7e4-6193-48d3-8bd3-c978a9a3d73d" (UID: "c232c7e4-6193-48d3-8bd3-c978a9a3d73d"). InnerVolumeSpecName "kube-api-access-xd6b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:41.834964 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.834924    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-maas-api-tls\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:41.834964 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.834955    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd6b7\" (UniqueName: \"kubernetes.io/projected/c232c7e4-6193-48d3-8bd3-c978a9a3d73d-kube-api-access-xd6b7\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:41.834964 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:41.834965    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpfwd\" (UniqueName: \"kubernetes.io/projected/75ebcf41-28fa-4df5-9636-c5c129209c46-kube-api-access-dpfwd\") on node \"ip-10-0-133-134.ec2.internal\" DevicePath \"\""
Jul 24 16:32:42.385693 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.385659    2567 generic.go:356] "Generic (PLEG): container finished" podID="75ebcf41-28fa-4df5-9636-c5c129209c46" containerID="17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5" exitCode=0
Jul 24 16:32:42.386194 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.385702    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"
Jul 24 16:32:42.386194 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.385752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" event={"ID":"75ebcf41-28fa-4df5-9636-c5c129209c46","Type":"ContainerDied","Data":"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"}
Jul 24 16:32:42.386194 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.385815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4" event={"ID":"75ebcf41-28fa-4df5-9636-c5c129209c46","Type":"ContainerDied","Data":"8d74c22fbbff066bd14e0a34a9903b5ed1121f5d78a139269228960bafa51a1b"}
Jul 24 16:32:42.386194 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.385849    2567 scope.go:117] "RemoveContainer" containerID="17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"
Jul 24 16:32:42.387391 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.387366    2567 generic.go:356] "Generic (PLEG): container finished" podID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" containerID="d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a" exitCode=0
Jul 24 16:32:42.387562 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.387420    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"
Jul 24 16:32:42.387562 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.387478    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" event={"ID":"c232c7e4-6193-48d3-8bd3-c978a9a3d73d","Type":"ContainerDied","Data":"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"}
Jul 24 16:32:42.387562 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.387526    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh" event={"ID":"c232c7e4-6193-48d3-8bd3-c978a9a3d73d","Type":"ContainerDied","Data":"97b4295073433e8a649b8b3f605a2e45f1891ea3010cda1153f000d60a37d500"}
Jul 24 16:32:42.398668 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.398650    2567 scope.go:117] "RemoveContainer" containerID="17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"
Jul 24 16:32:42.399002 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:32:42.398985    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5\": container with ID starting with 17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5 not found: ID does not exist" containerID="17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"
Jul 24 16:32:42.399051 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.399009    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5"} err="failed to get container status \"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5\": rpc error: code = NotFound desc = could not find container \"17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5\": container with ID starting with 17e12a5a7fe113ed920d7784a5de17770452a5023baa0a9a03111b3821eefaa5 not found: ID does not exist"
Jul 24 16:32:42.399051 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.399025    2567 scope.go:117] "RemoveContainer" containerID="d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"
Jul 24 16:32:42.407861 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.407823    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:32:42.410646 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.410622    2567 scope.go:117] "RemoveContainer" containerID="d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"
Jul 24 16:32:42.411217 ip-10-0-133-134 kubenswrapper[2567]: E0724 16:32:42.411195    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a\": container with ID starting with d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a not found: ID does not exist" containerID="d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"
Jul 24 16:32:42.411270 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.411229    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a"} err="failed to get container status \"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a\": rpc error: code = NotFound desc = could not find container \"d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a\": container with ID starting with d27149aed0a33eaa19977cbeee9987609ce07b6dfea8021dc9cc838dd27fbd1a not found: ID does not exist"
Jul 24 16:32:42.414642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.414616    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-5aebfb-69555bcfc6-6lnl4"]
Jul 24 16:32:42.425808 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.425758    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:32:42.427775 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.427749    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-5aebfb-5668c489f-t9lvh"]
Jul 24 16:32:42.949634 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.949598    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ebcf41-28fa-4df5-9636-c5c129209c46" path="/var/lib/kubelet/pods/75ebcf41-28fa-4df5-9636-c5c129209c46/volumes"
Jul 24 16:32:42.949984 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:42.949970    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" path="/var/lib/kubelet/pods/c232c7e4-6193-48d3-8bd3-c978a9a3d73d/volumes"
Jul 24 16:32:45.927416 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:45.927372    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"]
Jul 24 16:32:45.929826 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:45.929795    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-5aebfb-htgdw"]
Jul 24 16:32:45.935158 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:45.935117    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:46.348227 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:46.348191    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:46.948832 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:46.948798    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b24deec-4134-452e-b1a4-5815f9ada508" path="/var/lib/kubelet/pods/7b24deec-4134-452e-b1a4-5815f9ada508/volumes"
Jul 24 16:32:52.186255 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:52.186199    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:52.242746 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:52.242707    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:52.474459 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:52.474346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:52.714016 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:52.713963    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:53.438058 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:53.438005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:32:59.149235 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:59.149201    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64456c74cb-h6df5_0e0f3706-29cc-46a8-81fb-7b57d386114e/manager/0.log"
Jul 24 16:32:59.644442 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:59.644405    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6798866cc5-hmnrs_ec6adce5-79bb-445a-bda5-69cbbe11b0db/manager/0.log"
Jul 24 16:32:59.650968 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:32:59.650941    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6798866cc5-hmnrs_ec6adce5-79bb-445a-bda5-69cbbe11b0db/copy-manifests/0.log"
Jul 24 16:33:00.126547 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:00.126509    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-78bb94d85f-6pkr6_5af7435b-34c8-4d7a-a8b6-955fc0f65700/maas-api/0.log"
Jul 24 16:33:00.355453 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:00.355422    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748495-7xfsh_10907ab1-8bde-4795-80fb-419859397f5a/cleanup/0.log"
Jul 24 16:33:02.037794 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:02.037762    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kn4xq_5f084a21-e96c-43cd-af71-d0881e61caea/manager/0.log"
Jul 24 16:33:02.398968 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:02.398877    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fvbsf_e59191f3-f7c6-4054-b868-de0fc590df77/registry-server/0.log"
Jul 24 16:33:02.773874 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:02.773846    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-45972_cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e/manager/0.log"
Jul 24 16:33:03.253320 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:03.253293    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rr4wf_b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6/discovery/0.log"
Jul 24 16:33:03.365640 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:03.365611    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5688cff95c-j5665_1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f/kube-auth-proxy/0.log"
Jul 24 16:33:03.750189 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:03.750154    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bpc8w_96c89a27-1524-4016-9c7c-f03f7634db7e/payload-pre-processing/0.log"
Jul 24 16:33:04.314426 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:04.314393    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nhd4z_aaf50365-50b5-43c7-bd3d-e9cc8e15bbc5/main/0.log"
Jul 24 16:33:04.431179 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:04.431122    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tjgzz_a8e4e438-21d3-43a2-b8db-15f796a04a84/main/0.log"
Jul 24 16:33:04.782873 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:04.782837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hvtwk_2a8cc48a-0cfc-4458-bf68-97cc7d06af2c/main/0.log"
Jul 24 16:33:10.930947 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:10.930912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:33:11.003191 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:11.003161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log" limit=1024
Jul 24 16:33:13.677059 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:13.677030    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-59zdt_44fcb14e-69e3-4b41-8baa-ccb7b8f4793d/global-pull-secret-syncer/0.log"
Jul 24 16:33:13.771935 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:13.771905    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7wxwm_53a3261e-72d6-4de2-a75a-0256b5ee1513/konnectivity-agent/0.log"
Jul 24 16:33:13.888900 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:13.888861    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-134.ec2.internal_4fd32fde91ac02b5f7cfbf50c6d4bcc8/haproxy/0.log"
Jul 24 16:33:18.745559 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:18.745531    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kn4xq_5f084a21-e96c-43cd-af71-d0881e61caea/manager/0.log"
Jul 24 16:33:18.847265 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:18.847230    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fvbsf_e59191f3-f7c6-4054-b868-de0fc590df77/registry-server/0.log"
Jul 24 16:33:19.114739 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:19.114648    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-45972_cbd67c53-6da0-4ff2-a9c1-fd04d32efe7e/manager/0.log"
Jul 24 16:33:21.167866 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:21.167787    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4lls_c54c4da7-de32-4563-8729-ef83c39b3fb5/node-exporter/0.log"
Jul 24 16:33:21.190196 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:21.190172    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4lls_c54c4da7-de32-4563-8729-ef83c39b3fb5/kube-rbac-proxy/0.log"
Jul 24 16:33:21.210815 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:21.210791    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l4lls_c54c4da7-de32-4563-8729-ef83c39b3fb5/init-textfile/0.log"
Jul 24 16:33:23.795719 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.795682    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"]
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.795999    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b24deec-4134-452e-b1a4-5815f9ada508" containerName="revoke-keys"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796010    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b24deec-4134-452e-b1a4-5815f9ada508" containerName="revoke-keys"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796023    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ebcf41-28fa-4df5-9636-c5c129209c46" containerName="payload-pre-processing"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796030    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ebcf41-28fa-4df5-9636-c5c129209c46" containerName="payload-pre-processing"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796045    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" containerName="maas-api"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796051    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" containerName="maas-api"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796099    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b24deec-4134-452e-b1a4-5815f9ada508" containerName="revoke-keys"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796107    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c232c7e4-6193-48d3-8bd3-c978a9a3d73d" containerName="maas-api"
Jul 24 16:33:23.796224 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.796114    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ebcf41-28fa-4df5-9636-c5c129209c46" containerName="payload-pre-processing"
Jul 24 16:33:23.798992 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.798970    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:23.801294 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.801276    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7s9sw\"/\"openshift-service-ca.crt\""
Jul 24 16:33:23.802213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.802194    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7s9sw\"/\"default-dockercfg-fh7j9\""
Jul 24 16:33:23.802286 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.802194    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7s9sw\"/\"kube-root-ca.crt\""
Jul 24 16:33:23.807788 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.807765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"]
Jul 24 16:33:23.908291 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.908256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfgfk\" (UniqueName: \"kubernetes.io/projected/eed9a9d4-b561-4c46-b0a0-92728aa80321-kube-api-access-xfgfk\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:23.908478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.908325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-sys\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:23.908478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.908345    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-proc\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:23.908478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.908388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-podres\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:23.908478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:23.908407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-lib-modules\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009344 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfgfk\" (UniqueName: \"kubernetes.io/projected/eed9a9d4-b561-4c46-b0a0-92728aa80321-kube-api-access-xfgfk\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-sys\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-proc\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-podres\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-lib-modules\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-sys\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009544 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-proc\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009588    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-lib-modules\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.009778 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.009603    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eed9a9d4-b561-4c46-b0a0-92728aa80321-podres\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.017716 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.017683    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfgfk\" (UniqueName: \"kubernetes.io/projected/eed9a9d4-b561-4c46-b0a0-92728aa80321-kube-api-access-xfgfk\") pod \"perf-node-gather-daemonset-gdjt9\" (UID: \"eed9a9d4-b561-4c46-b0a0-92728aa80321\") " pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.108988 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.108895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.247478 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.247449    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"]
Jul 24 16:33:24.250037 ip-10-0-133-134 kubenswrapper[2567]: W0724 16:33:24.250002    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeed9a9d4_b561_4c46_b0a0_92728aa80321.slice/crio-68be06beb9880cab2611cf9eae60bb8042e273fa1d4b01d6086460555ea32af3 WatchSource:0}: Error finding container 68be06beb9880cab2611cf9eae60bb8042e273fa1d4b01d6086460555ea32af3: Status 404 returned error can't find the container with id 68be06beb9880cab2611cf9eae60bb8042e273fa1d4b01d6086460555ea32af3
Jul 24 16:33:24.587542 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.587510    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9" event={"ID":"eed9a9d4-b561-4c46-b0a0-92728aa80321","Type":"ContainerStarted","Data":"8f1b06fe89eb215e2efcb8f8b68f57bb2d37e139719726706b01d1c0e04715d0"}
Jul 24 16:33:24.587718 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.587563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9" event={"ID":"eed9a9d4-b561-4c46-b0a0-92728aa80321","Type":"ContainerStarted","Data":"68be06beb9880cab2611cf9eae60bb8042e273fa1d4b01d6086460555ea32af3"}
Jul 24 16:33:24.587718 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.587592    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:24.605959 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:24.605906    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9" podStartSLOduration=1.605890068 podStartE2EDuration="1.605890068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:33:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:33:24.604826059 +0000 UTC m=+3146.157065828" watchObservedRunningTime="2026-07-24 16:33:24.605890068 +0000 UTC m=+3146.158129841"
Jul 24 16:33:26.024443 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:26.024413    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2cf7_9a52f9e9-355e-4e1e-93ab-757fbaee85d6/dns/0.log"
Jul 24 16:33:26.047143 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:26.047101    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2cf7_9a52f9e9-355e-4e1e-93ab-757fbaee85d6/kube-rbac-proxy/0.log"
Jul 24 16:33:26.188435 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:26.188402    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kt2bk_445e9c17-f645-469d-b0a1-1807c3e9a08e/dns-node-resolver/0.log"
Jul 24 16:33:26.716773 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:26.716738    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-85b877574b-4k6sp_4ddca80d-ac16-4302-a00b-1449186e6c96/registry/0.log"
Jul 24 16:33:26.783633 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:26.783598    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkglg_40a3161d-caec-4cb8-bdb9-cbaa63c8dcd5/node-ca/0.log"
Jul 24 16:33:27.819932 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:27.819895    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rr4wf_b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6/discovery/0.log"
Jul 24 16:33:27.839536 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:27.839506    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5688cff95c-j5665_1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f/kube-auth-proxy/0.log"
Jul 24 16:33:27.954381 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:27.954350    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bpc8w_96c89a27-1524-4016-9c7c-f03f7634db7e/payload-pre-processing/0.log"
Jul 24 16:33:28.795574 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:28.795539    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-w87bk_54d7cb73-8e96-4b36-ac2e-9e0d8bc48eab/serve-healthcheck-canary/0.log"
Jul 24 16:33:29.358817 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:29.358770    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkfp2_62ba8453-57e4-4f6f-aee0-73dd75259434/kube-rbac-proxy/0.log"
Jul 24 16:33:29.374572 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:29.374524    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkfp2_62ba8453-57e4-4f6f-aee0-73dd75259434/exporter/0.log"
Jul 24 16:33:29.394820 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:29.394788    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkfp2_62ba8453-57e4-4f6f-aee0-73dd75259434/extractor/0.log"
Jul 24 16:33:30.602463 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:30.602433    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7s9sw/perf-node-gather-daemonset-gdjt9"
Jul 24 16:33:31.832896 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:31.832866    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64456c74cb-h6df5_0e0f3706-29cc-46a8-81fb-7b57d386114e/manager/0.log"
Jul 24 16:33:32.059183 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:32.059151    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6798866cc5-hmnrs_ec6adce5-79bb-445a-bda5-69cbbe11b0db/manager/0.log"
Jul 24 16:33:32.079995 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:32.079967    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6798866cc5-hmnrs_ec6adce5-79bb-445a-bda5-69cbbe11b0db/copy-manifests/0.log"
Jul 24 16:33:33.363207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:33.363176    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ztp9q_1d9d192d-5116-4d6b-bc2a-125bef5dd963/openshift-lws-operator/0.log"
Jul 24 16:33:34.144551 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:34.144512    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rr4wf_b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6/discovery/0.log"
Jul 24 16:33:34.154852 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:34.154823    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5688cff95c-j5665_1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f/kube-auth-proxy/0.log"
Jul 24 16:33:34.243342 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:34.243314    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bpc8w_96c89a27-1524-4016-9c7c-f03f7634db7e/payload-pre-processing/0.log"
Jul 24 16:33:35.871744 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:35.871713    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rr4wf_b0d4fb3f-caf4-4a3d-b6a5-dd5d5269eec6/discovery/0.log"
Jul 24 16:33:35.892000 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:35.891971    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5688cff95c-j5665_1c6d8bde-deb3-4bb1-ae5a-cfa24ec2d36f/kube-auth-proxy/0.log"
Jul 24 16:33:35.986401 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:35.986365    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-bpc8w_96c89a27-1524-4016-9c7c-f03f7634db7e/payload-pre-processing/0.log"
Jul 24 16:33:38.493642 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:38.493606    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m7lh8_fd79b9a9-991e-46a5-b5b0-2cc6710106f0/migrator/0.log"
Jul 24 16:33:38.513882 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:38.513850    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-m7lh8_fd79b9a9-991e-46a5-b5b0-2cc6710106f0/graceful-termination/0.log"
Jul 24 16:33:39.785213 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.785180    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4nkwr_42c1cc67-5c1b-46b3-b120-1d89c93e832e/kube-multus/0.log"
Jul 24 16:33:39.807297 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.807271    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/kube-multus-additional-cni-plugins/0.log"
Jul 24 16:33:39.828014 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.827988    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/egress-router-binary-copy/0.log"
Jul 24 16:33:39.847976 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.847933    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/cni-plugins/0.log"
Jul 24 16:33:39.869110 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.869074    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/bond-cni-plugin/0.log"
Jul 24 16:33:39.888809 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.888783    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/routeoverride-cni/0.log"
Jul 24 16:33:39.908652 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.908624    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/whereabouts-cni-bincopy/0.log"
Jul 24 16:33:39.928097 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:39.928063    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-69fgt_37801522-98d0-4c13-a06a-2a700a32d141/whereabouts-cni/0.log"
Jul 24 16:33:40.498360 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:40.498336    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jbfbj_f7afd27a-11a7-4701-9b08-408dcc75cc08/network-metrics-daemon/0.log"
Jul 24 16:33:40.531182 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:40.531156    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jbfbj_f7afd27a-11a7-4701-9b08-408dcc75cc08/kube-rbac-proxy/0.log"
Jul 24 16:33:41.390715 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.390678    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/ovn-controller/0.log"
Jul 24 16:33:41.420207 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.420172    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/ovn-acl-logging/0.log"
Jul 24 16:33:41.454820 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.454774    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/kube-rbac-proxy-node/0.log"
Jul 24 16:33:41.498896 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.498866    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/kube-rbac-proxy-ovn-metrics/0.log"
Jul 24 16:33:41.542538 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.542498    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/northd/0.log"
Jul 24 16:33:41.581998 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.581960    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/nbdb/0.log"
Jul 24 16:33:41.620869 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.620837    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/sbdb/0.log"
Jul 24 16:33:41.701480 ip-10-0-133-134 kubenswrapper[2567]: I0724 16:33:41.701388    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-89rdl_d8bfae1f-8145-47cd-8531-da65a17409e3/ovnkube-controller/0.log"