Jul 29 20:46:25.546111 ip-10-0-139-241 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 20:46:25.907853 ip-10-0-139-241 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:46:25.907853 ip-10-0-139-241 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 20:46:25.907853 ip-10-0-139-241 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:46:25.907853 ip-10-0-139-241 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 20:46:25.907853 ip-10-0-139-241 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 20:46:25.908457 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.907918    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 20:46:25.913648 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913634    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.913648 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913647    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.913939 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913920    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913942    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913951    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913957    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913965    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913972    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913976    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913981    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913985    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913990    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913994    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.913998    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914008    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914012    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914017    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.914011 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914021    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914026    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914030    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914035    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914039    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914044    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914049    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914053    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914057    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914066    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914070    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914074    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914078    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914083    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914086    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914099    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914103    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914107    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914114    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.914570 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914123    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914130    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914139    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914143    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914148    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914152    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914156    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914160    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914164    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914168    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914172    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914177    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914181    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914185    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914189    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914198    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914202    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914207    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914211    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914215    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.915309 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914219    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914224    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914229    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914234    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914238    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914242    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914246    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914255    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914260    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914289    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914294    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914298    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914302    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914306    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914310    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914314    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914318    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914322    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914327    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914338    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.915804 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914347    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914351    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914355    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914362    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914366    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914370    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914374    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914379    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914384    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914388    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914392    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914400    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914404    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914409    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914413    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914418    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914422    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914426    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914430    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914434    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.916285 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914438    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914442    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.914469    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915761    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915770    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915774    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915777    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915780    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915782    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915785    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915788    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915790    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915793    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915795    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915797    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915800    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915802    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915805    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915807    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.916760 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915812    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915815    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915817    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915820    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915823    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915825    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915828    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915830    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915833    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915835    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915840    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915844    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915848    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915852    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915855    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915858    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915861    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915864    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915866    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.917270 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915885    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915887    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915890    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915893    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915895    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915898    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915900    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915903    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915906    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915908    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915911    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915913    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915916    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915919    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915922    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915924    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915927    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915929    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915932    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915934    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.917725 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915937    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915939    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915941    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915944    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915947    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915949    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915952    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915954    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915957    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915960    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915962    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915964    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915967    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915969    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915972    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915974    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915977    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915980    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915982    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915984    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.918209 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915987    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915989    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915992    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915995    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.915997    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916000    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916004    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916007    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916009    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916013    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916021    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916024    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916026    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916028    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916031    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916033    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916036    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916038    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916040    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916043    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.918686 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916045    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916048    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916051    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916053    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916430    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916435    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916438    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916441    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916443    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916447    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916451    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916454    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916456    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916459    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916461    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916464    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916466    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916469    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916472    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.919190 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916477    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916480    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916483    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916486    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916488    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916491    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916493    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916495    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916498    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916500    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916504    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916506    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916508    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916511    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916513    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916516    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916518    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916521    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916523    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916526    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.919630 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916528    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916531    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916533    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916536    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916538    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916541    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916544    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916546    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916549    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916551    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916554    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916558    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916561    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916564    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916567    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916570    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916573    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916576    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916579    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.920107 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916581    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916584    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916586    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916589    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916592    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916594    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916598    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916600    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916603    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916605    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916608    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916610    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916613    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916615    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916618    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916620    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916624    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916627    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916630    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.920543 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916633    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916635    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916638    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916641    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916643    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916646    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916649    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916651    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916653    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916656    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916658    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916661    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916663    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916666    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916668    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916671    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916673    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916676    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916678    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916680    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.921044 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916683    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916685    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916687    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916690    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916692    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.916695    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916755    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916768    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916773    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916778    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916783    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916788    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916794    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916800    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916805    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916812    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916816    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916820    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916823    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916825    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916828    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916831    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916834    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 20:46:25.921509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916837    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916841    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916844    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916847    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916850    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916853    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916857    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916859    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916862    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916865    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916879    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916882    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916885    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916888    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916891    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916895    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916899    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916902    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916905    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916908    2569 flags.go:64] FLAG: --enable-server="true"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916911    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916914    2569 flags.go:64] FLAG: --event-burst="100"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916917    2569 flags.go:64] FLAG: --event-qps="50"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916920    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 20:46:25.922087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916924    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916927    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916931    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916933    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916937    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916939    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916942    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916945    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916948    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916950    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916954    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916957    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916960    2569 flags.go:64] FLAG: --feature-gates=""
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916963    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916966    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916969    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916972    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916975    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916978    2569 flags.go:64] FLAG: --help="false"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916981    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916984    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916987    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916989    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916993    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 20:46:25.922688 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.916999    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917001    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917004    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917007    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917010    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917013    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917015    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917018    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917022    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917025    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917028    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917031    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917033    2569 flags.go:64] FLAG: --lock-file=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917036    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917039    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917042    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917047    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917049    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917052    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917055    2569 flags.go:64] FLAG: --logging-format="text"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917058    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917061    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917064    2569 flags.go:64] FLAG: --manifest-url=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917067    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917071    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 20:46:25.923300 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917073    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917077    2569 flags.go:64] FLAG: --max-pods="110"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917080    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917083    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917085    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917088    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917091    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917095    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917098    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917105    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917108    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917111    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917114    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917117    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917122    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917126    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917129    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917132    2569 flags.go:64] FLAG: --port="10250"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917135    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917138    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-010d11d2dec44367c"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917141    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917144    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917146    2569 flags.go:64] FLAG: --register-node="true"
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917149    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 29 20:46:25.923908 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917152    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917155    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917158    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917161    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917165    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917167    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917170    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917173    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917176    2569 flags.go:64] FLAG: --runonce="false"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917178    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917181    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917184    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917187    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917189    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917192    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917195    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917199    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917202    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917204    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917207    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917209    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917212    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917215    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917219    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917224    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 20:46:25.924490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917227    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917230    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917234    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917237    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917239    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917242    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917245    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917248    2569 flags.go:64] FLAG: --v="2"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917252    2569 flags.go:64] FLAG: --version="false"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917258    2569 flags.go:64] FLAG: --vmodule=""
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917261    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.917266    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917376    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917380    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917383    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917386    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917388    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917391    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917393    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917396    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917398    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917401    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.925097 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917404    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917407    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917410    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917414    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917418    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917421    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917424    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917426    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917430    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917433    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917436    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917439    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917442    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917445    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917448    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917452    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917456    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917459    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917461    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.925658 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917464    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917466    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917469    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917472    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917475    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917477    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917480    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917482    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917484    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917487    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917490    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917492    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917494    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917497    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917500    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917503    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917505    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917508    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917510    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917513    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.926120 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917515    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917518    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917521    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917523    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917526    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917528    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917531    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917533    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917536    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917538    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917541    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917543    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917546    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917549    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917551    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917554    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917557    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917559    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917561    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917564    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.926598 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917566    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917569    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917571    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917574    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917576    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917578    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917582    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917584    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917587    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917589    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917592    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917595    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917597    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917602    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917604    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917607    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917609    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917612    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917614    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917617    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.927220 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917619    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917622    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917624    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917626    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917629    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917631    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917634    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917636    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917639    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.917641    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.918199    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.924424    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.924438    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.924450    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 20:46:25.927693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.924455    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924510    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924514    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924518    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924521    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924524    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924527    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924530    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924534    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924537    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924541    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924543    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924546    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924549    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924551    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924554    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924557    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924560    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924562    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.928053 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924566    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924570    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924573    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924576    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924578    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924581    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924584    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924587    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924589    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924592    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924595    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924599    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924601    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924604    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924606    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924609    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924611    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924613    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924616    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924618    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.928492 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924621    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924623    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924626    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924629    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924631    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924633    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924636    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924639    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924641    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924644    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924646    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924649    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924651    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924654    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924656    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924659    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924661    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924664    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924666    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924669    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.928970 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924671    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924674    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924676    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924679    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924685    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924688    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924690    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924693    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924695    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924698    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924700    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924703    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924705    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924707    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924710    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924712    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924715    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924717    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924719    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924722    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.929432 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924725    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924729    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924731    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924734    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924736    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924738    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924741    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924743    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924746    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924748    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924750    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924753    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924755    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924758    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924761    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924764    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924766    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924770    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924772    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.929908 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924775    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924777    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.924782    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924917    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924921    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924923    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924926    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924928    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924931    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924934    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924936    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924939    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924941    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924943    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924946    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 20:46:25.930364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924949    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924951    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924953    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924956    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924958    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924961    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924963    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924966    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924968    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924971    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924973    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924975    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924979    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924981    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924984    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924986    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924995    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.924998    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925000    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925002    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 20:46:25.930727 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925005    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925007    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925010    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925012    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925014    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925017    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925020    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925022    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925024    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925027    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925029    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925031    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925034    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925036    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925038    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925041    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925043    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925045    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925048    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925050    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 20:46:25.931240 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925052    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925055    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925057    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925060    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925062    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925064    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925067    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925069    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925072    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925080    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925082    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925085    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925087    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925089    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925092    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925094    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925096    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925099    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925101    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925104    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 20:46:25.931704 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925106    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925109    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925112    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925114    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925118    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925121    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925124    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925127    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925130    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925132    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925135    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925138    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925140    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925142    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925145    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925147    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925150    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925152    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925155    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 20:46:25.932249 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925159    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925162    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925164    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925173    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925176    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925179    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925182    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:25.925185    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.925190    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.925298    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.927125    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.928634    2569 server.go:1019] "Starting client certificate rotation"
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.928757    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 20:46:25.932683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.928802    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 20:46:25.933028 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.931136    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 20:46:25.933028 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.932814    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 20:46:25.944955 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.944925    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 29 20:46:25.946828 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.946815    2569 log.go:25] "Validated CRI v1 image API"
Jul 29 20:46:25.948469 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.948449    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 20:46:25.950970 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.950952    2569 fs.go:135] Filesystem UUIDs: map[3e93e347-8edc-4e36-a2d2-96d3fd610ba7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e9ff1558-2cd9-4cf2-9feb-02f5727cc31f:/dev/nvme0n1p4]
Jul 29 20:46:25.951032 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.950969    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 20:46:25.956113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.956018    2569 manager.go:216] Machine: {Timestamp:2026-07-29 20:46:25.954666954 +0000 UTC m=+0.299507597 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199791 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2133c2920eb17c993bae6dc69b25d6 SystemUUID:ec2133c2-920e-b17c-993b-ae6dc69b25d6 BootID:2c27208b-af07-4775-9114-282fc630e026 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:00:6a:44:bc:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:00:6a:44:bc:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:18:70:5a:92:31 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 20:46:25.956113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.956111    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 20:46:25.956242 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.956225    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 20:46:25.956384 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.956368    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 20:46:25.960494 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.960468    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 20:46:25.960618 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.960497    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 20:46:25.961192 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.961181    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 20:46:25.961223 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.961194    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 20:46:25.961223 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.961201    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 20:46:25.961718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.961709    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 20:46:25.962453 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.962443    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 20:46:25.964191 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964180    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 20:46:25.964305 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964296    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 20:46:25.964339 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964323    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 20:46:25.964339 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964331    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 20:46:25.964391 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964346    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 20:46:25.964391 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964358    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 29 20:46:25.964391 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964372    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 20:46:25.964753 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964740    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 20:46:25.964800 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.964760    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 20:46:25.968013 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.967992    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 20:46:25.970240 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.970227    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 20:46:25.970287 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.970256    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 20:46:25.971992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.971978    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 20:46:25.972037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.971999    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 20:46:25.972037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972008    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 20:46:25.972037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972016    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 20:46:25.972037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972027    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 20:46:25.972037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972035    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972046    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972053    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972062    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972070    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972080    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 20:46:25.972163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972087    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 20:46:25.972728 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972717    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 20:46:25.972764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.972729    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 20:46:25.974525 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.974503    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 20:46:25.974614 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.974527    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 20:46:25.975658 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.975647    2569 server.go:1268] "Started kubelet"
Jul 29 20:46:25.976041 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.976006    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 20:46:25.976178 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.976145    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 20:46:25.976260 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.976168    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 20:46:25.976651 ip-10-0-139-241 systemd[1]: Started Kubernetes Kubelet.
Jul 29 20:46:25.979024 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.978221    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 20:46:25.979672 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.979656    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 20:46:25.981692 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.981671    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h78cs"
Jul 29 20:46:25.982108 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.982088    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 20:46:25.983757 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.983743    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 20:46:25.984942 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.983995    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-241.ec2.internal.18c6dda106fbf5d1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-241.ec2.internal,UID:ip-10-0-139-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-241.ec2.internal,},FirstTimestamp:2026-07-29 20:46:25.975629265 +0000 UTC m=+0.320469908,LastTimestamp:2026-07-29 20:46:25.975629265 +0000 UTC m=+0.320469908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-241.ec2.internal,}"
Jul 29 20:46:25.985389 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.985373    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 20:46:25.985444 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.985377    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 20:46:25.985563 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.985544    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:25.986086 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986068    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 20:46:25.986219 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986174    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 20:46:25.986219 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986192    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 20:46:25.986359 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986277    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 20:46:25.986359 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986288    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 20:46:25.986359 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986322    2569 factory.go:55] Registering systemd factory
Jul 29 20:46:25.986359 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986342    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 29 20:46:25.986568 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986554    2569 factory.go:153] Registering CRI-O factory
Jul 29 20:46:25.986618 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986571    2569 factory.go:223] Registration of the crio container factory successfully
Jul 29 20:46:25.986665 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986619    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 20:46:25.986665 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986642    2569 factory.go:103] Registering Raw factory
Jul 29 20:46:25.986665 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.986655    2569 manager.go:1196] Started watching for new ooms in manager
Jul 29 20:46:25.987241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.987102    2569 manager.go:349] Starting recovery of all containers
Jul 29 20:46:25.989015 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.988992    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h78cs"
Jul 29 20:46:25.996181 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.996157    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:25.998854 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:25.998837    2569 manager.go:354] Recovery completed
Jul 29 20:46:25.999246 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:25.999229    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-241.ec2.internal\" not found" node="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.004087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.004073    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.005836 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.005817    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.005909 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.005843    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.005909 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.005858    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.006460 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.006441    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 20:46:26.006460 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.006455    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 20:46:26.006616 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.006476    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 20:46:26.006616 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.006485    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.007256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.007241    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.007335 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.007277    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.007335 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.007290    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.008489 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.008478    2569 policy_none.go:49] "None policy: Start"
Jul 29 20:46:26.008522 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.008493    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 20:46:26.008522 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.008502    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 20:46:26.009377 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.009367    2569 policy_none.go:47] "Start"
Jul 29 20:46:26.037600 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.037501    2569 manager.go:337] "Starting Device Plugin manager"
Jul 29 20:46:26.037678 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.037628    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 20:46:26.037678 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.037639    2569 server.go:87] "Starting device plugin registration server"
Jul 29 20:46:26.037845 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.037834    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 20:46:26.037915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.037846    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 20:46:26.037961 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.037927    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 20:46:26.038161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.038003    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 20:46:26.038161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.038015    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 20:46:26.039908 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.039888    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 20:46:26.039980 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.039940    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.138143 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.138125    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.139395 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.139380    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.139467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.139408    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.139467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.139417    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.139467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.139439    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.143314 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.143289    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 20:46:26.144429 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.144408    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 20:46:26.144502 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.144441    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 20:46:26.144502 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.144458    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 20:46:26.144502 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.144487    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 20:46:26.145456 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.145442    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.145556 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.145461    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-241.ec2.internal\": node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.147602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.147586    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:26.226854 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.226814    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.244919 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.244901    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"]
Jul 29 20:46:26.244973 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.244966    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.246909 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.246897    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.246980 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.246921    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.246980 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.246935    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.248117 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248105    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.248254 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.248291 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248267    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.248784 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248769    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.248841 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248797    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.248841 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248806    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.248932 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248769    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.248932 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248887    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.248932 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.248904    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.249884 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.249858    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.249936 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.249894    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 20:46:26.250507 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.250493    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 20:46:26.250559 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.250519    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 20:46:26.250559 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.250534    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeHasSufficientPID"
Jul 29 20:46:26.272490 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.272472    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-241.ec2.internal\" not found" node="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.276608 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.276593    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-241.ec2.internal\" not found" node="ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.288022 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.288005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.288095 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.288028    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.288095 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.288051    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9347e8bed064fb1726f4031c3510ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-139-241.ec2.internal\" (UID: \"9347e8bed064fb1726f4031c3510ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.327073 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.327054    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.388202 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.388293 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.388293 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9347e8bed064fb1726f4031c3510ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-139-241.ec2.internal\" (UID: \"9347e8bed064fb1726f4031c3510ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.388293 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.388293 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388256    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9347e8bed064fb1726f4031c3510ca65-config\") pod \"kube-apiserver-proxy-ip-10-0-139-241.ec2.internal\" (UID: \"9347e8bed064fb1726f4031c3510ca65\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.388429 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.388294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dc4374f68b5c1cdf14190d471fe5db5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal\" (UID: \"5dc4374f68b5c1cdf14190d471fe5db5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.427309 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.427290    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.528074 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:26.528028    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:26.575191 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.575170    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:26.580529 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:26.580514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:28.375811 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:28.375789    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:28.476297 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:28.476274    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:28.576724 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:28.576682    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:28.675203 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.675182    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 20:46:28.675566 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.675317    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 20:46:28.675566 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.675337    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 20:46:28.677325 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:28.677311    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:28.740444 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.740409    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 20:41:27 +0000 UTC" deadline="2028-03-29 01:34:16.841568324 +0000 UTC"
Jul 29 20:46:28.740444 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.740438    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14596h47m48.101133083s"
Jul 29 20:46:28.777691 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:28.777671    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-241.ec2.internal\" not found"
Jul 29 20:46:28.795401 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.795382    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:28.833471 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.833417    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:28.843833 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.843817    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 20:46:28.845341 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.845325    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal"
Jul 29 20:46:28.851024 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.851010    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 20:46:28.888900 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:28.888872    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9347e8bed064fb1726f4031c3510ca65.slice/crio-f3d0562d9a594129ea423d99f05de07496f37b3dd0debb2fa98f4eaa1e72f850 WatchSource:0}: Error finding container f3d0562d9a594129ea423d99f05de07496f37b3dd0debb2fa98f4eaa1e72f850: Status 404 returned error can't find the container with id f3d0562d9a594129ea423d99f05de07496f37b3dd0debb2fa98f4eaa1e72f850
Jul 29 20:46:28.890317 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:28.890294    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc4374f68b5c1cdf14190d471fe5db5.slice/crio-e514c6cd4a39c7f7605d7187881ed7db544b50f8d6fe487e5bef87c2b34f3d49 WatchSource:0}: Error finding container e514c6cd4a39c7f7605d7187881ed7db544b50f8d6fe487e5bef87c2b34f3d49: Status 404 returned error can't find the container with id e514c6cd4a39c7f7605d7187881ed7db544b50f8d6fe487e5bef87c2b34f3d49
Jul 29 20:46:28.894527 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.894482    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal" event={"ID":"5dc4374f68b5c1cdf14190d471fe5db5","Type":"ContainerStarted","Data":"e514c6cd4a39c7f7605d7187881ed7db544b50f8d6fe487e5bef87c2b34f3d49"}
Jul 29 20:46:28.894612 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.894603    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:46:28.895589 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.895571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal" event={"ID":"9347e8bed064fb1726f4031c3510ca65","Type":"ContainerStarted","Data":"f3d0562d9a594129ea423d99f05de07496f37b3dd0debb2fa98f4eaa1e72f850"}
Jul 29 20:46:28.896872 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:28.896854    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:29.551592 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.551565    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:29.711750 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.711711    2569 apiserver.go:52] "Watching apiserver"
Jul 29 20:46:29.720422 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.720397    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 20:46:29.721705 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.721682    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-sljj6","kube-system/global-pull-secret-syncer-m96cs","kube-system/konnectivity-agent-c5v7z","kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal","openshift-dns/node-resolver-wx7q5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal","openshift-multus/multus-additional-cni-plugins-fkswq","openshift-multus/network-metrics-daemon-jj8sp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb","openshift-cluster-node-tuning-operator/tuned-dt6n8","openshift-image-registry/node-ca-j96sc","openshift-multus/multus-grzwl","openshift-network-diagnostics/network-check-target-lpmln","openshift-network-operator/iptables-alerter-vbn27"]
Jul 29 20:46:29.723279 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.723256    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.723364 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.723332    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:29.724484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.724460    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.725971 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.725950    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.727127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.726997    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.727127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727005    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 20:46:29.727127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727056    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.727127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727101    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hp4td\""
Jul 29 20:46:29.728018 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727993    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.728018 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727997    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.728168 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.727998    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9w8sf\""
Jul 29 20:46:29.728306 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.728289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.728396 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.728375    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 20:46:29.728877 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.728857    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 20:46:29.729638 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.729622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.730290 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.730272    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 20:46:29.730384 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.730363    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-snkvd\""
Jul 29 20:46:29.730921 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.730901    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.731246 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731226    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 20:46:29.731324 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731288    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.731422 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731405    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 20:46:29.731497 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731483    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 20:46:29.731590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731574    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.731696 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.731696 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731690    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.731838 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.731758    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5wkml\""
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.732851    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733023    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vjl7x\""
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733147    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733259    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733319    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.733484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733355    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 20:46:29.733856 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.733810    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.736358 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.736335    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 20:46:29.736447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.736392    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mcwvv\""
Jul 29 20:46:29.736503 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.736393    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 20:46:29.737376 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.737352    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.737538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.737518    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 20:46:29.737697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.737682    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.738756 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.738721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.739054 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.739025    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7vbpq\""
Jul 29 20:46:29.741018 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.740996    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.741456 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.741443    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:46:29.741834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.741818    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 20:46:29.741928 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.741914    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fprvm\""
Jul 29 20:46:29.742679 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.742664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:29.742781 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.742718    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:29.743038 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.743022    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-s4jsw\""
Jul 29 20:46:29.743103 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.743073    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 20:46:29.743298 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.743284    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 20:46:29.744073 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.744151 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.744112    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:29.744622 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744602    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 20:46:29.744708 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744679    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd9lv\" (UniqueName: \"kubernetes.io/projected/ad1052f7-0f92-4807-a179-014bf646b54f-kube-api-access-cd9lv\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.744788 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744707    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bd68cab-02ad-41ee-a519-6a4aec969e38-iptables-alerter-script\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.744788 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744731    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.744788 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cnibin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-system-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-socket-dir-parent\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-daemon-config\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8lrx\" (UniqueName: \"kubernetes.io/projected/fd012239-a602-4f38-a26e-ecdaad610b62-kube-api-access-s8lrx\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.744930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g6br\" (UniqueName: \"kubernetes.io/projected/404e431a-29b9-473e-a5b4-b270fb83dff7-kube-api-access-7g6br\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.744980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-os-release\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d35b2869-d157-4f59-a05e-3dd14b1ff33e-hosts-file\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745023    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-device-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-config\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-node-log\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-netd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745179 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745159    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-script-lib\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745194    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-os-release\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745241    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-socket-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745289    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-bin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-hostroot\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-multus-certs\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745350    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35b2869-d157-4f59-a05e-3dd14b1ff33e-tmp-dir\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745370    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-netns\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sls8\" (UniqueName: \"kubernetes.io/projected/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-kube-api-access-6sls8\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhzjd\" (UniqueName: \"kubernetes.io/projected/e0bafcad-96c6-4c92-9594-8f00bc6164ce-kube-api-access-hhzjd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745429    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-var-lib-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745472    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-env-overrides\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovn-node-metrics-cert\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745510    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-k8s-cni-cncf-io\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745530    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-kubelet\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-bin\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745603    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-conf-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd012239-a602-4f38-a26e-ecdaad610b62-host\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745654    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd012239-a602-4f38-a26e-ecdaad610b62-serviceca\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745692    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl7rh\" (UniqueName: \"kubernetes.io/projected/3bd68cab-02ad-41ee-a519-6a4aec969e38-kube-api-access-gl7rh\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-etc-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-systemd-units\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-log-socket\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.745985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745839    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cnibin\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745879    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhswc\" (UniqueName: \"kubernetes.io/projected/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-kube-api-access-nhswc\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bd68cab-02ad-41ee-a519-6a4aec969e38-host-slash\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-kubelet\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745944    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-ovn\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cni-binary-copy\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.745993    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-etc-kubernetes\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-registration-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-sys-fs\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746047    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-slash\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746062    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbvf9\" (UniqueName: \"kubernetes.io/projected/d35b2869-d157-4f59-a05e-3dd14b1ff33e-kube-api-access-dbvf9\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-systemd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746102    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-netns\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.746602 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-multus\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.747374 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.746188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-system-cni-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.786910 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.786893    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l8cvs"
Jul 29 20:46:29.796791 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.796773    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l8cvs"
Jul 29 20:46:29.834159 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.834110    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 20:46:29.846457 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8lrx\" (UniqueName: \"kubernetes.io/projected/fd012239-a602-4f38-a26e-ecdaad610b62-kube-api-access-s8lrx\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.846558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846474    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.846558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.846558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846520    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6br\" (UniqueName: \"kubernetes.io/projected/404e431a-29b9-473e-a5b4-b270fb83dff7-kube-api-access-7g6br\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.846558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-var-lib-kubelet\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-os-release\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d35b2869-d157-4f59-a05e-3dd14b1ff33e-hosts-file\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-device-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846653    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846653    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-os-release\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysconfig\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846697    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d35b2869-d157-4f59-a05e-3dd14b1ff33e-hosts-file\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846714    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-device-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.846764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-config\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec015c0d-0ae9-4d65-97e5-6f54258963e1-konnectivity-ca\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846793    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-node-log\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846816    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-netd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-node-log\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-script-lib\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-os-release\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-socket-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-os-release\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846972    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-bin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-bin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-hostroot\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847041    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-hostroot\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847028    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-socket-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-multus-certs\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-multus-certs\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35b2869-d157-4f59-a05e-3dd14b1ff33e-tmp-dir\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.847153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-modprobe-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-kubernetes\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-sys\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-netns\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sls8\" (UniqueName: \"kubernetes.io/projected/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-kube-api-access-6sls8\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.846894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-netd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d35b2869-d157-4f59-a05e-3dd14b1ff33e-tmp-dir\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w68fh\" (UniqueName: \"kubernetes.io/projected/08b38a0a-5365-4250-a648-31a582ab1121-kube-api-access-w68fh\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhzjd\" (UniqueName: \"kubernetes.io/projected/e0bafcad-96c6-4c92-9594-8f00bc6164ce-kube-api-access-hhzjd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-var-lib-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-netns\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-kubelet-config\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.847493    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-env-overrides\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.847969 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.847563    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:46:30.347543963 +0000 UTC m=+2.945520317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovn-node-metrics-cert\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-k8s-cni-cncf-io\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-var-lib-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847623    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-kubelet\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847679    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-k8s-cni-cncf-io\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847700    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-kubelet\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847713    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-dbus\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847716    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-config\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-run\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847764    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovnkube-script-lib\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847807    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-bin\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-conf-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847872    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-cni-bin\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd012239-a602-4f38-a26e-ecdaad610b62-host\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847895    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 20:46:29.848697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847889    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-conf-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd012239-a602-4f38-a26e-ecdaad610b62-serviceca\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848028    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e0bafcad-96c6-4c92-9594-8f00bc6164ce-env-overrides\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.847939    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fd012239-a602-4f38-a26e-ecdaad610b62-host\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-host\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-etc-tuned\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl7rh\" (UniqueName: \"kubernetes.io/projected/3bd68cab-02ad-41ee-a519-6a4aec969e38-kube-api-access-gl7rh\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-etc-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-systemd-units\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-log-socket\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848325    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fd012239-a602-4f38-a26e-ecdaad610b62-serviceca\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cnibin\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-etc-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.849467 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848445    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848479    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-systemd-units\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhswc\" (UniqueName: \"kubernetes.io/projected/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-kube-api-access-nhswc\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848520    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bd68cab-02ad-41ee-a519-6a4aec969e38-host-slash\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-kubelet\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848962    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849262    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-log-socket\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.848421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cnibin\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849431    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849439    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3bd68cab-02ad-41ee-a519-6a4aec969e38-host-slash\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-ovn\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cni-binary-copy\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849727    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec015c0d-0ae9-4d65-97e5-6f54258963e1-agent-certs\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-conf\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849821    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-etc-kubernetes\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-registration-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.850215 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-sys-fs\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849948    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-sys-fs\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850012    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-etc-kubernetes\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/404e431a-29b9-473e-a5b4-b270fb83dff7-registration-dir\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-ovn\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-lib-modules\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-slash\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbvf9\" (UniqueName: \"kubernetes.io/projected/d35b2869-d157-4f59-a05e-3dd14b1ff33e-kube-api-access-dbvf9\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-systemd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cni-binary-copy\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850385    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-openvswitch\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-slash\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850483    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-netns\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-multus\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.850992 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-system-cni-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-systemd\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850651    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd9lv\" (UniqueName: \"kubernetes.io/projected/ad1052f7-0f92-4807-a179-014bf646b54f-kube-api-access-cd9lv\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850680    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bd68cab-02ad-41ee-a519-6a4aec969e38-iptables-alerter-script\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850702    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-run-systemd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850771    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cnibin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-var-lib-cni-multus\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850795    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-tmp\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-host-run-netns\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-system-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-socket-dir-parent\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-daemon-config\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851386    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3bd68cab-02ad-41ee-a519-6a4aec969e38-iptables-alerter-script\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851393    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-daemon-config\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.850888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-system-cni-dir\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.851716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851475    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-system-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.852464 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851526    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-socket-dir-parent\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.852464 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851527    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-multus-cni-dir\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.852464 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851579    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-cnibin\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.852464 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.851656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e0bafcad-96c6-4c92-9594-8f00bc6164ce-ovn-node-metrics-cert\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.852631 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.849511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0bafcad-96c6-4c92-9594-8f00bc6164ce-host-kubelet\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.855321 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.855300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8lrx\" (UniqueName: \"kubernetes.io/projected/fd012239-a602-4f38-a26e-ecdaad610b62-kube-api-access-s8lrx\") pod \"node-ca-j96sc\" (UID: \"fd012239-a602-4f38-a26e-ecdaad610b62\") " pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:29.857457 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.857437    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6br\" (UniqueName: \"kubernetes.io/projected/404e431a-29b9-473e-a5b4-b270fb83dff7-kube-api-access-7g6br\") pod \"aws-ebs-csi-driver-node-vjnvb\" (UID: \"404e431a-29b9-473e-a5b4-b270fb83dff7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:29.860113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.860093    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sls8\" (UniqueName: \"kubernetes.io/projected/17ee39ab-9224-4efb-9bc6-83ce76a3f90e-kube-api-access-6sls8\") pod \"multus-grzwl\" (UID: \"17ee39ab-9224-4efb-9bc6-83ce76a3f90e\") " pod="openshift-multus/multus-grzwl"
Jul 29 20:46:29.860491 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.860471    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhzjd\" (UniqueName: \"kubernetes.io/projected/e0bafcad-96c6-4c92-9594-8f00bc6164ce-kube-api-access-hhzjd\") pod \"ovnkube-node-sljj6\" (UID: \"e0bafcad-96c6-4c92-9594-8f00bc6164ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:29.864140 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.864119    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl7rh\" (UniqueName: \"kubernetes.io/projected/3bd68cab-02ad-41ee-a519-6a4aec969e38-kube-api-access-gl7rh\") pod \"iptables-alerter-vbn27\" (UID: \"3bd68cab-02ad-41ee-a519-6a4aec969e38\") " pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:29.867369 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.867350    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbvf9\" (UniqueName: \"kubernetes.io/projected/d35b2869-d157-4f59-a05e-3dd14b1ff33e-kube-api-access-dbvf9\") pod \"node-resolver-wx7q5\" (UID: \"d35b2869-d157-4f59-a05e-3dd14b1ff33e\") " pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:29.867454 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.867411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhswc\" (UniqueName: \"kubernetes.io/projected/48e16bae-b350-46ad-bd4c-c5c7e288ebf1-kube-api-access-nhswc\") pod \"multus-additional-cni-plugins-fkswq\" (UID: \"48e16bae-b350-46ad-bd4c-c5c7e288ebf1\") " pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:29.867497 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.867465    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd9lv\" (UniqueName: \"kubernetes.io/projected/ad1052f7-0f92-4807-a179-014bf646b54f-kube-api-access-cd9lv\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:29.951470 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951450    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.951585 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-systemd\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951585 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-tmp\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951585 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951513    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-var-lib-kubelet\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951585 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysconfig\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec015c0d-0ae9-4d65-97e5-6f54258963e1-konnectivity-ca\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951609    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysconfig\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951634    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-systemd\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-var-lib-kubelet\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-modprobe-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-modprobe-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.951728    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:29.951775 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-kubernetes\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951795    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-sys\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-kubernetes\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.951829    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:30.451814171 +0000 UTC m=+3.049790508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951856    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-sys\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w68fh\" (UniqueName: \"kubernetes.io/projected/08b38a0a-5365-4250-a648-31a582ab1121-kube-api-access-w68fh\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-kubelet-config\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951956    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-kubelet-config\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-d\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.951981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-dbus\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952025    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-run\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-host\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952076    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-etc-tuned\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952130    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2a65e073-18d7-43d6-a600-eab03235eff9-dbus\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec015c0d-0ae9-4d65-97e5-6f54258963e1-agent-certs\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952149    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-run\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ec015c0d-0ae9-4d65-97e5-6f54258963e1-konnectivity-ca\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952166    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-conf\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-host\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-lib-modules\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-etc-sysctl-conf\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.952876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.952331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08b38a0a-5365-4250-a648-31a582ab1121-lib-modules\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.953940 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.953912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-tmp\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.954152 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.954136    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08b38a0a-5365-4250-a648-31a582ab1121-etc-tuned\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:29.954618 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.954596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ec015c0d-0ae9-4d65-97e5-6f54258963e1-agent-certs\") pod \"konnectivity-agent-c5v7z\" (UID: \"ec015c0d-0ae9-4d65-97e5-6f54258963e1\") " pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:29.957772 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.957753    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:29.957855 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.957775    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:29.957855 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.957789    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:29.957855 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:29.957853    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:30.457836376 +0000 UTC m=+3.055812728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:29.959819 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:29.959800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w68fh\" (UniqueName: \"kubernetes.io/projected/08b38a0a-5365-4250-a648-31a582ab1121-kube-api-access-w68fh\") pod \"tuned-dt6n8\" (UID: \"08b38a0a-5365-4250-a648-31a582ab1121\") " pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:30.036851 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.036828    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vbn27"
Jul 29 20:46:30.044584 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.044563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-grzwl"
Jul 29 20:46:30.052069 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.052046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:30.058471 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.058456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wx7q5"
Jul 29 20:46:30.064461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.064442    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j96sc"
Jul 29 20:46:30.070820 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.070795    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fkswq"
Jul 29 20:46:30.071938 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.071923    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 20:46:30.076764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.076744    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb"
Jul 29 20:46:30.082675 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.082648    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dt6n8"
Jul 29 20:46:30.087564 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.087543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:30.355835 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.355769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:30.355979 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.355946    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:30.356041 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.356009    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:46:31.35599249 +0000 UTC m=+3.953968829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:30.456466 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.456440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:30.456607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.456543    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:30.456607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.456598    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:31.456583562 +0000 UTC m=+4.054559902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:30.557621 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.557583    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:30.557801 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.557777    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:30.557851 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.557803    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:30.557851 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.557815    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:30.557967 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.557873    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:31.557855534 +0000 UTC m=+4.155831897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:30.623150 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.623122    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b38a0a_5365_4250_a648_31a582ab1121.slice/crio-10937d115f9609796aadd98d0fe9faca62b8573510e2d03faffa3a0e6ac2ee8f WatchSource:0}: Error finding container 10937d115f9609796aadd98d0fe9faca62b8573510e2d03faffa3a0e6ac2ee8f: Status 404 returned error can't find the container with id 10937d115f9609796aadd98d0fe9faca62b8573510e2d03faffa3a0e6ac2ee8f
Jul 29 20:46:30.624206 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.624182    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404e431a_29b9_473e_a5b4_b270fb83dff7.slice/crio-a3c3dc53b190a9413f266f31fa550840e2738a50446ed53d49bd9c2801d9b8f6 WatchSource:0}: Error finding container a3c3dc53b190a9413f266f31fa550840e2738a50446ed53d49bd9c2801d9b8f6: Status 404 returned error can't find the container with id a3c3dc53b190a9413f266f31fa550840e2738a50446ed53d49bd9c2801d9b8f6
Jul 29 20:46:30.624848 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.624823    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bafcad_96c6_4c92_9594_8f00bc6164ce.slice/crio-1220ea94ad5a3ea892be7364261687ccf318ec4147aa998def3d3d1ea5bb58a4 WatchSource:0}: Error finding container 1220ea94ad5a3ea892be7364261687ccf318ec4147aa998def3d3d1ea5bb58a4: Status 404 returned error can't find the container with id 1220ea94ad5a3ea892be7364261687ccf318ec4147aa998def3d3d1ea5bb58a4
Jul 29 20:46:30.628008 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.627987    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ee39ab_9224_4efb_9bc6_83ce76a3f90e.slice/crio-19fd06c181a5aef7dce879183e935efd5fe59035356d414ad543f1d0dde43894 WatchSource:0}: Error finding container 19fd06c181a5aef7dce879183e935efd5fe59035356d414ad543f1d0dde43894: Status 404 returned error can't find the container with id 19fd06c181a5aef7dce879183e935efd5fe59035356d414ad543f1d0dde43894
Jul 29 20:46:30.628955 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.628928    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48e16bae_b350_46ad_bd4c_c5c7e288ebf1.slice/crio-341a1fbb077a4f6a1a07347bf486bf3a5985e7fe4ebc087e538e2b948a73eb15 WatchSource:0}: Error finding container 341a1fbb077a4f6a1a07347bf486bf3a5985e7fe4ebc087e538e2b948a73eb15: Status 404 returned error can't find the container with id 341a1fbb077a4f6a1a07347bf486bf3a5985e7fe4ebc087e538e2b948a73eb15
Jul 29 20:46:30.629576 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.629509    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd68cab_02ad_41ee_a519_6a4aec969e38.slice/crio-bb963e3c22510dcd8e8ca874d16630364c40ddb77239f48a90702e5f3318e821 WatchSource:0}: Error finding container bb963e3c22510dcd8e8ca874d16630364c40ddb77239f48a90702e5f3318e821: Status 404 returned error can't find the container with id bb963e3c22510dcd8e8ca874d16630364c40ddb77239f48a90702e5f3318e821
Jul 29 20:46:30.630893 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.630856    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd012239_a602_4f38_a26e_ecdaad610b62.slice/crio-3f7ae6d6f7003d036b92b41aeb1dc1a6d0a533c61f92ec9677ac58a6238f2d0e WatchSource:0}: Error finding container 3f7ae6d6f7003d036b92b41aeb1dc1a6d0a533c61f92ec9677ac58a6238f2d0e: Status 404 returned error can't find the container with id 3f7ae6d6f7003d036b92b41aeb1dc1a6d0a533c61f92ec9677ac58a6238f2d0e
Jul 29 20:46:30.631148 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.631124    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35b2869_d157_4f59_a05e_3dd14b1ff33e.slice/crio-0150e985515ce5af89844b342261f011e563ce54301e0e33c6a5b33b541a3f0b WatchSource:0}: Error finding container 0150e985515ce5af89844b342261f011e563ce54301e0e33c6a5b33b541a3f0b: Status 404 returned error can't find the container with id 0150e985515ce5af89844b342261f011e563ce54301e0e33c6a5b33b541a3f0b
Jul 29 20:46:30.632859 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:46:30.632807    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec015c0d_0ae9_4d65_97e5_6f54258963e1.slice/crio-fa7ca44479ba37d6684dc52299abee564c01fcd2c0ca1c659d916b14229d9378 WatchSource:0}: Error finding container fa7ca44479ba37d6684dc52299abee564c01fcd2c0ca1c659d916b14229d9378: Status 404 returned error can't find the container with id fa7ca44479ba37d6684dc52299abee564c01fcd2c0ca1c659d916b14229d9378
Jul 29 20:46:30.799507 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.799317    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:41:29 +0000 UTC" deadline="2028-05-11 21:31:23.641861274 +0000 UTC"
Jul 29 20:46:30.799903 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.799508    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15648h44m52.842358829s"
Jul 29 20:46:30.891564 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.891531    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:30.891658 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:30.891641    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:30.899076 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.899052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wx7q5" event={"ID":"d35b2869-d157-4f59-a05e-3dd14b1ff33e","Type":"ContainerStarted","Data":"0150e985515ce5af89844b342261f011e563ce54301e0e33c6a5b33b541a3f0b"}
Jul 29 20:46:30.900042 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.900022    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vbn27" event={"ID":"3bd68cab-02ad-41ee-a519-6a4aec969e38","Type":"ContainerStarted","Data":"bb963e3c22510dcd8e8ca874d16630364c40ddb77239f48a90702e5f3318e821"}
Jul 29 20:46:30.901008 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.900971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerStarted","Data":"341a1fbb077a4f6a1a07347bf486bf3a5985e7fe4ebc087e538e2b948a73eb15"}
Jul 29 20:46:30.901919 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.901891    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grzwl" event={"ID":"17ee39ab-9224-4efb-9bc6-83ce76a3f90e","Type":"ContainerStarted","Data":"19fd06c181a5aef7dce879183e935efd5fe59035356d414ad543f1d0dde43894"}
Jul 29 20:46:30.902869 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.902852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dt6n8" event={"ID":"08b38a0a-5365-4250-a648-31a582ab1121","Type":"ContainerStarted","Data":"10937d115f9609796aadd98d0fe9faca62b8573510e2d03faffa3a0e6ac2ee8f"}
Jul 29 20:46:30.904463 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.904445    2569 generic.go:356] "Generic (PLEG): container finished" podID="5dc4374f68b5c1cdf14190d471fe5db5" containerID="c7e7675329d2f16b91c79c7af18f46865f4cc7f0f048a5f25749f5d54eeab4b6" exitCode=0
Jul 29 20:46:30.904539 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.904501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal" event={"ID":"5dc4374f68b5c1cdf14190d471fe5db5","Type":"ContainerDied","Data":"c7e7675329d2f16b91c79c7af18f46865f4cc7f0f048a5f25749f5d54eeab4b6"}
Jul 29 20:46:30.905620 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.905600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j96sc" event={"ID":"fd012239-a602-4f38-a26e-ecdaad610b62","Type":"ContainerStarted","Data":"3f7ae6d6f7003d036b92b41aeb1dc1a6d0a533c61f92ec9677ac58a6238f2d0e"}
Jul 29 20:46:30.906653 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.906629    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"1220ea94ad5a3ea892be7364261687ccf318ec4147aa998def3d3d1ea5bb58a4"}
Jul 29 20:46:30.907618 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.907588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb" event={"ID":"404e431a-29b9-473e-a5b4-b270fb83dff7","Type":"ContainerStarted","Data":"a3c3dc53b190a9413f266f31fa550840e2738a50446ed53d49bd9c2801d9b8f6"}
Jul 29 20:46:30.908773 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.908753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal" event={"ID":"9347e8bed064fb1726f4031c3510ca65","Type":"ContainerStarted","Data":"d31bc6fdddd0d4abd87bd4caabf4d3a0d314d66a63bdafbbbc7018a66e1498f2"}
Jul 29 20:46:30.909774 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.909758    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5v7z" event={"ID":"ec015c0d-0ae9-4d65-97e5-6f54258963e1","Type":"ContainerStarted","Data":"fa7ca44479ba37d6684dc52299abee564c01fcd2c0ca1c659d916b14229d9378"}
Jul 29 20:46:30.932380 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:30.932338    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-241.ec2.internal" podStartSLOduration=2.932326042 podStartE2EDuration="2.932326042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:30.932102651 +0000 UTC m=+3.530079006" watchObservedRunningTime="2026-07-29 20:46:30.932326042 +0000 UTC m=+3.530302398"
Jul 29 20:46:31.367310 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.367278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:31.367488 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.367470    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:31.367568 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.367537    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:46:33.367517046 +0000 UTC m=+5.965493406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:31.468007 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.467967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:31.468171 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.468147    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:31.468230 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.468205    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:33.468185649 +0000 UTC m=+6.066161986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:31.568528 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.568497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:31.568754 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.568724    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:31.568836 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.568763    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:31.568836 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.568776    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:31.568836 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.568829    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:33.568810467 +0000 UTC m=+6.166786816 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:31.799812 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.799748    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 20:41:29 +0000 UTC" deadline="2028-05-07 15:37:09.022506052 +0000 UTC"
Jul 29 20:46:31.799812 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.799783    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15546h50m37.222727702s"
Jul 29 20:46:31.894768 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.894250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:31.894768 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.894372    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:31.894768 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.894464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:31.894768 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:31.894541    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:31.928776 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.928725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal" event={"ID":"5dc4374f68b5c1cdf14190d471fe5db5","Type":"ContainerStarted","Data":"36984a2d99deca879907302ed86f0a4f732313dd5caa08c854ef299c46d8b883"}
Jul 29 20:46:31.941981 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:31.941932    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-241.ec2.internal" podStartSLOduration=3.941915345 podStartE2EDuration="3.941915345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:31.941673089 +0000 UTC m=+4.539649445" watchObservedRunningTime="2026-07-29 20:46:31.941915345 +0000 UTC m=+4.539891701"
Jul 29 20:46:32.892257 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:32.892227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:32.892761 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:32.892364    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:33.384642 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:33.384537    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:33.384818 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.384754    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:33.384818 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.384815    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:46:37.3847963 +0000 UTC m=+9.982772637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:33.485433 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:33.485336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:33.485588 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.485506    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:33.485588 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.485579    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:37.485556579 +0000 UTC m=+10.083532914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:33.586749 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:33.586691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:33.586916 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.586871    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:33.586916 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.586891    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:33.586916 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.586903    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:33.587063 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.586958    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:37.586940128 +0000 UTC m=+10.184916476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:33.892161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:33.892126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:33.892316 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.892256    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:33.892316 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:33.892279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:33.892749 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:33.892380    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:34.892310 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:34.891863    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:34.892310 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:34.891992    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:35.892239 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:35.892210    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:35.892381 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:35.892325    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:35.892928 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:35.892777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:35.892928 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:35.892890    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:36.892538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:36.892505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:36.892943 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:36.892657    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:37.416514 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:37.416001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:37.416514 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.416138    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:37.416514 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.416200    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:46:45.416180245 +0000 UTC m=+18.014156592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:37.516558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:37.516244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:37.516558 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.516409    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:37.516558 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.516464    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:45.51644548 +0000 UTC m=+18.114421819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:37.616593 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:37.616562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:37.616777 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.616755    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:37.616880 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.616786    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:37.616880 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.616800    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:37.616880 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.616856    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:46:45.616837118 +0000 UTC m=+18.214813479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:37.893671 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:37.893240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:37.893671 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:37.893267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:37.893671 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.893352    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:37.893671 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:37.893436    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:38.891586 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:38.891516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:38.891773 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:38.891660    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:39.891783 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:39.891755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:39.892316 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:39.891755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:39.892316 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:39.891887    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:39.892316 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:39.891956    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:40.892260 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:40.892190    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:40.892686 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:40.892299    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:41.892047 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:41.891986    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:41.892233 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:41.892116    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:41.892233 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:41.892155    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:41.892233 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:41.892220    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:42.892036 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:42.892000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:42.892199 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:42.892120    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:43.891751 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:43.891706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:43.892198 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:43.891851    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:43.892198 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:43.891924    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:43.892198 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:43.892004    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:44.892146 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:44.892114    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:44.892649 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:44.892254    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:45.480573 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:45.480540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:45.480748 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.480700    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:45.480792 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.480779    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:47:01.480762761 +0000 UTC m=+34.078739096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:46:45.581760 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:45.581714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:45.581892 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.581840    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:45.581946 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.581900    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:01.581885368 +0000 UTC m=+34.179861702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:46:45.682452 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:45.682422    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:45.682607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.682528    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:46:45.682607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.682549    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:46:45.682607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.682559    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:45.682607 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.682604    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:01.682590632 +0000 UTC m=+34.280566967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:46:45.891950 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:45.891876    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:45.892112 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:45.891876    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:45.892112 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.892012    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:45.892112 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:45.892088    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:46.891843 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:46.891808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:46.892026 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:46.891933    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:47.892816 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:47.892788    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:47.893256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:47.892842    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:47.893256 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:47.892938    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:47.893256 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:47.893043    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:48.892432 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.892334    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:48.892532 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:48.892511    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:48.974392 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.974363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5v7z" event={"ID":"ec015c0d-0ae9-4d65-97e5-6f54258963e1","Type":"ContainerStarted","Data":"ded8882d6351fa0dd08b9fb777cbf07d7af8f54150f75cdbf85b89b4441ceded"}
Jul 29 20:46:48.976181 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.976155    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wx7q5" event={"ID":"d35b2869-d157-4f59-a05e-3dd14b1ff33e","Type":"ContainerStarted","Data":"14cd2df7cb07f77310204f2cd03c3d8dc03f5e82d66ceb88af97fccd277cb121"}
Jul 29 20:46:48.978216 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.978191    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="10dc24c19cc8208adff3d8b5ed3890bbf1a786f64e43c4f7c0bc84b97af34762" exitCode=0
Jul 29 20:46:48.978301 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.978258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"10dc24c19cc8208adff3d8b5ed3890bbf1a786f64e43c4f7c0bc84b97af34762"}
Jul 29 20:46:48.980033 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.980009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-grzwl" event={"ID":"17ee39ab-9224-4efb-9bc6-83ce76a3f90e","Type":"ContainerStarted","Data":"4dfaf99055defa7cf4743212fa856cd55b4c148314f495951421bad2437e9600"}
Jul 29 20:46:48.981784 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.981761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dt6n8" event={"ID":"08b38a0a-5365-4250-a648-31a582ab1121","Type":"ContainerStarted","Data":"88a09a74003b01297c0ab3c5ec3391380c30b694982a2c9928e9e5278ca22e87"}
Jul 29 20:46:48.983165 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.983144    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j96sc" event={"ID":"fd012239-a602-4f38-a26e-ecdaad610b62","Type":"ContainerStarted","Data":"32ff9bbef792dbb91d444514dfbdff6c872941b5d508135f78668e9c128b2d3d"}
Jul 29 20:46:48.986435 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.986412    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"a09e25d6ac2963c075717a85337795b3ba8d855e9714ae8e9b449bfbef8fbc56"}
Jul 29 20:46:48.986501 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.986488    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"621722dcd1bad2439287b3cc0cc2436381d03bebaba499ff7b0b2e72b57372d5"}
Jul 29 20:46:48.986547 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.986535    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"b6774de17a67f9af9a82156b6ddbe4a908e0f54899944fae59eff88403d220a6"}
Jul 29 20:46:48.986596 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.986585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"1b6e7fb57025594cebcf4bcacc48bf046f999e4fb7cfbe3bddb012d57b08691b"}
Jul 29 20:46:48.987580 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.987546    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c5v7z" podStartSLOduration=11.061512516 podStartE2EDuration="20.987520597s" totalImagesPullingTime="9.926008081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:48.987488176 +0000 UTC m=+21.585464552" watchObservedRunningTime="2026-07-29 20:46:48.987520597 +0000 UTC m=+21.585496931"
Jul 29 20:46:48.988712 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:48.988666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb" event={"ID":"404e431a-29b9-473e-a5b4-b270fb83dff7","Type":"ContainerStarted","Data":"697565b5222aa84667c3b74348fd8dfe3c457ed503d0c5d229d30edca5922f20"}
Jul 29 20:46:49.000176 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.000138    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j96sc" podStartSLOduration=4.338842184 podStartE2EDuration="22.00012681s" totalImagesPullingTime="17.661284626s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:48.999569688 +0000 UTC m=+21.597546044" watchObservedRunningTime="2026-07-29 20:46:49.00012681 +0000 UTC m=+21.598103223"
Jul 29 20:46:49.035457 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.035417    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-grzwl" podStartSLOduration=4.3341296 podStartE2EDuration="22.035406237s" totalImagesPullingTime="17.701276637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:49.034982134 +0000 UTC m=+21.632958490" watchObservedRunningTime="2026-07-29 20:46:49.035406237 +0000 UTC m=+21.633382593"
Jul 29 20:46:49.049100 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.049016    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wx7q5" podStartSLOduration=4.388121349 podStartE2EDuration="22.049002723s" totalImagesPullingTime="17.660881374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:49.04871789 +0000 UTC m=+21.646694245" watchObservedRunningTime="2026-07-29 20:46:49.049002723 +0000 UTC m=+21.646979075"
Jul 29 20:46:49.065579 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.065547    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dt6n8" podStartSLOduration=3.394835966 podStartE2EDuration="21.065536852s" totalImagesPullingTime="17.670700886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:49.065303273 +0000 UTC m=+21.663279628" watchObservedRunningTime="2026-07-29 20:46:49.065536852 +0000 UTC m=+21.663513208"
Jul 29 20:46:49.778853 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.778812    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 20:46:49.797173 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.796899    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T20:46:49.778830525Z","UUID":"ee6a601f-a225-4287-979f-ae59f9aebd9c","Handler":null,"Name":"","Endpoint":""}
Jul 29 20:46:49.798562 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.798543    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 20:46:49.798670 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.798583    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 20:46:49.891808 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.891731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:49.891933 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:49.891868    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:49.891997 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.891933    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:49.892083 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:49.892045    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:49.992473 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.992434    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vbn27" event={"ID":"3bd68cab-02ad-41ee-a519-6a4aec969e38","Type":"ContainerStarted","Data":"dd5ee8d77ea75f5d8cf2cd4670dea1b3851c9e4f1a4da621d87ca8fd40d3ff9a"}
Jul 29 20:46:49.995695 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.995673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"ffdf563f876266e2c160ab76c0acac7b13222d737a211afc143c7497cdcde62f"}
Jul 29 20:46:49.995830 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.995750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"ed5d593d8bcd77c9af369591faa6774cb350b4d874267c04927bbf024c5185ad"}
Jul 29 20:46:49.997663 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:49.997641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb" event={"ID":"404e431a-29b9-473e-a5b4-b270fb83dff7","Type":"ContainerStarted","Data":"e039307d316c5394e03af25c26c75efd8ca8eefacd9b34185b0fc4413c5f1f4a"}
Jul 29 20:46:50.007142 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:50.007095    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vbn27" podStartSLOduration=5.345103596 podStartE2EDuration="23.007082912s" totalImagesPullingTime="17.661979316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:50.006937928 +0000 UTC m=+22.604914285" watchObservedRunningTime="2026-07-29 20:46:50.007082912 +0000 UTC m=+22.605059268"
Jul 29 20:46:50.615287 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:50.615259    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:50.615985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:50.615952    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:50.891944 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:50.891763    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:50.892074 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:50.892053    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:51.002482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:51.002428    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb" event={"ID":"404e431a-29b9-473e-a5b4-b270fb83dff7","Type":"ContainerStarted","Data":"aea8584b92c449c3dd0ee0ae084a434323f0ea70da68dde39f8b279b192077dd"}
Jul 29 20:46:51.018990 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:51.018949    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjnvb" podStartSLOduration=3.169896907 podStartE2EDuration="23.01893645s" totalImagesPullingTime="19.849039543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:51.018727098 +0000 UTC m=+23.616703456" watchObservedRunningTime="2026-07-29 20:46:51.01893645 +0000 UTC m=+23.616912805"
Jul 29 20:46:51.891724 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:51.891689    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:51.891931 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:51.891813    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:51.891931 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:51.891866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:51.892044 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:51.891978    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:52.008301 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:52.008264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"dd46d774dd7fece84f6d2ae74e1a2ab4064090800176787eb12525f30f5625cf"}
Jul 29 20:46:52.008935 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:52.008515    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 20:46:52.891985 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:52.891954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:52.892145 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:52.892058    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:53.892458 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:53.892309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:53.892981 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:53.892383    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:53.892981 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:53.892524    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:53.892981 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:53.892603    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:54.013600 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.013575    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="1b6501c3fd1583207db3b555eed55167837cc6b2407be139e7cd9375ca5fc39f" exitCode=0
Jul 29 20:46:54.013703 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.013600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"1b6501c3fd1583207db3b555eed55167837cc6b2407be139e7cd9375ca5fc39f"}
Jul 29 20:46:54.021787 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.021769    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" event={"ID":"e0bafcad-96c6-4c92-9594-8f00bc6164ce","Type":"ContainerStarted","Data":"af891bd4507ff2acd783cf0ba70f0c70f27c1d92fd1395193087ec76c6da5b97"}
Jul 29 20:46:54.025075 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.022430    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:54.025075 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.022520    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:54.025075 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.022580    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:54.038324 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.038308    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:54.038416 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.038399    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:46:54.054860 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.054825    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6" podStartSLOduration=9.136235597 podStartE2EDuration="27.054813121s" totalImagesPullingTime="17.918577524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:46:54.054552088 +0000 UTC m=+26.652528443" watchObservedRunningTime="2026-07-29 20:46:54.054813121 +0000 UTC m=+26.652789476"
Jul 29 20:46:54.891788 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:54.891762    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:54.891918 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:54.891893    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:55.469997 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.469823    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jj8sp"]
Jul 29 20:46:55.470317 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.470036    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:55.470317 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:55.470145    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:55.472554 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.472533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m96cs"]
Jul 29 20:46:55.472654 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.472622    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:55.472710 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:55.472695    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:55.487205 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.487143    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpmln"]
Jul 29 20:46:55.487307 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:55.487234    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:55.487307 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:55.487301    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:56.027679 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.027649    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="b4cca6d3f1ecd615c1f578bb84b5b7697a8a5184c6354869df88bfc602722c70" exitCode=0
Jul 29 20:46:56.027823 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.027701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"b4cca6d3f1ecd615c1f578bb84b5b7697a8a5184c6354869df88bfc602722c70"}
Jul 29 20:46:56.648752 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.648713    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:56.649186 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.648864    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 20:46:56.649284 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.649270    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c5v7z"
Jul 29 20:46:56.892306 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.892284    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:56.892426 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:56.892284    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:56.892426 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:56.892371    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:56.892497 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:56.892432    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:57.032656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:57.032631    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerStarted","Data":"79efd32c2c9d99ee7f122eefb7cf759801a794917e0af249a866bc0d899f344d"}
Jul 29 20:46:57.893070 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:57.893044    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:57.893604 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:57.893121    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:46:58.037792 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:58.037757    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="79efd32c2c9d99ee7f122eefb7cf759801a794917e0af249a866bc0d899f344d" exitCode=0
Jul 29 20:46:58.037913 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:58.037837    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"79efd32c2c9d99ee7f122eefb7cf759801a794917e0af249a866bc0d899f344d"}
Jul 29 20:46:58.891699 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:58.891664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:46:58.891699 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:58.891683    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:46:58.891941 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:58.891792    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:46:58.891941 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:58.891864    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:46:59.892567 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:46:59.892496    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:46:59.893098 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:46:59.892614    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lpmln" podUID="9e1927f3-8f82-4454-a39e-dc8f4e9719d9"
Jul 29 20:47:00.830252 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:00.830220    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:00.891991 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:00.891961    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:00.892226 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:00.891964    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:00.892226 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:00.892089    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m96cs" podUID="2a65e073-18d7-43d6-a600-eab03235eff9"
Jul 29 20:47:00.892226 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:00.892179    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jj8sp" podUID="ad1052f7-0f92-4807-a179-014bf646b54f"
Jul 29 20:47:00.917870 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:00.917843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:00.975016 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:00.974984    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:01.246134 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.246110    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-241.ec2.internal" event="NodeReady"
Jul 29 20:47:01.246295 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.246221    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 20:47:01.293483 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.293452    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r8v9m"]
Jul 29 20:47:01.297707 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.297685    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sfgfh"]
Jul 29 20:47:01.297839 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.297812    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.300207 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.300188    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-79k6t\""
Jul 29 20:47:01.300315 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.300225    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 20:47:01.300485 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.300461    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.300828 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.300812    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 20:47:01.302935 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.302865    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 20:47:01.302935 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.302933    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 20:47:01.303146 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.302942    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 20:47:01.303146 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.303005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cc48x\""
Jul 29 20:47:01.305348 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.305300    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sfgfh"]
Jul 29 20:47:01.310644 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.310619    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8v9m"]
Jul 29 20:47:01.406570 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-tmp-dir\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.406705 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-config-volume\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.406705 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs7tj\" (UniqueName: \"kubernetes.io/projected/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-kube-api-access-cs7tj\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.406705 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7jh\" (UniqueName: \"kubernetes.io/projected/03c2ee14-ab98-46df-96f7-312d4fd0b652-kube-api-access-2j7jh\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.406873 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c2ee14-ab98-46df-96f7-312d4fd0b652-cert\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.406873 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.406769    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.507918 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.507859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-tmp-dir\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.507918 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.507897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-config-volume\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.507918 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.507914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs7tj\" (UniqueName: \"kubernetes.io/projected/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-kube-api-access-cs7tj\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.508108 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7jh\" (UniqueName: \"kubernetes.io/projected/03c2ee14-ab98-46df-96f7-312d4fd0b652-kube-api-access-2j7jh\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.508108 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508079    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c2ee14-ab98-46df-96f7-312d4fd0b652-cert\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.508168 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508107    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.508168 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508123    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:01.508248 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.508230    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:01.508289 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508252    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-tmp-dir\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.508336 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.508289    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs podName:ad1052f7-0f92-4807-a179-014bf646b54f nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.508269513 +0000 UTC m=+66.106245847 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs") pod "network-metrics-daemon-jj8sp" (UID: "ad1052f7-0f92-4807-a179-014bf646b54f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 20:47:01.508461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.508443    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-config-volume\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.508538 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.508521    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:01.508591 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.508559    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:02.008547966 +0000 UTC m=+34.606524301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:01.512482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.512344    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03c2ee14-ab98-46df-96f7-312d4fd0b652-cert\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.516054 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.516035    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7jh\" (UniqueName: \"kubernetes.io/projected/03c2ee14-ab98-46df-96f7-312d4fd0b652-kube-api-access-2j7jh\") pod \"ingress-canary-sfgfh\" (UID: \"03c2ee14-ab98-46df-96f7-312d4fd0b652\") " pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.516139 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.516079    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs7tj\" (UniqueName: \"kubernetes.io/projected/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-kube-api-access-cs7tj\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:01.608902 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.608876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:01.609031 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.608994    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:01.609093 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.609052    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret podName:2a65e073-18d7-43d6-a600-eab03235eff9 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.609036424 +0000 UTC m=+66.207012764 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret") pod "global-pull-secret-syncer-m96cs" (UID: "2a65e073-18d7-43d6-a600-eab03235eff9") : object "kube-system"/"original-pull-secret" not registered
Jul 29 20:47:01.618970 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.618942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sfgfh"
Jul 29 20:47:01.709333 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.709309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:01.709512 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.709491    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 20:47:01.709560 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.709519    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 20:47:01.709560 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.709533    2569 projected.go:196] Error preparing data for projected volume kube-api-access-zwsqt for pod openshift-network-diagnostics/network-check-target-lpmln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:01.709619 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:01.709599    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt podName:9e1927f3-8f82-4454-a39e-dc8f4e9719d9 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.709580123 +0000 UTC m=+66.307556465 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zwsqt" (UniqueName: "kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt") pod "network-check-target-lpmln" (UID: "9e1927f3-8f82-4454-a39e-dc8f4e9719d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 20:47:01.892803 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.892325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:01.896129 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.895459    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 20:47:01.896129 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.895516    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zcmz5\""
Jul 29 20:47:01.896129 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.895673    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 20:47:01.957172 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:01.957147    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sfgfh"]
Jul 29 20:47:01.960998 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:01.960973    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c2ee14_ab98_46df_96f7_312d4fd0b652.slice/crio-f334136ecfaaf1df163d16baed939db5cace0e2365fb1c1848681fb5482455b4 WatchSource:0}: Error finding container f334136ecfaaf1df163d16baed939db5cace0e2365fb1c1848681fb5482455b4: Status 404 returned error can't find the container with id f334136ecfaaf1df163d16baed939db5cace0e2365fb1c1848681fb5482455b4
Jul 29 20:47:02.012362 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.012326    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:02.012481 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:02.012457    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:02.012540 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:02.012517    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:03.012502928 +0000 UTC m=+35.610479266 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:02.047048 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.047013    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sfgfh" event={"ID":"03c2ee14-ab98-46df-96f7-312d4fd0b652","Type":"ContainerStarted","Data":"f334136ecfaaf1df163d16baed939db5cace0e2365fb1c1848681fb5482455b4"}
Jul 29 20:47:02.049896 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.049870    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerStarted","Data":"784bc2cdfae61c086bd0eee3328edb705f98aebd1b2462df14c8d0492f032c7b"}
Jul 29 20:47:02.781866 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.781785    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:02.883551 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.883517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:02.892527 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.892504    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:02.892651 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.892511    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:02.895137 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.895108    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 20:47:02.895767 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.895746    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 20:47:02.895861 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.895749    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lzlv4\""
Jul 29 20:47:02.983944 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:02.983919    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:03.019948 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.019923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:03.020089 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:03.020066    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:03.020188 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:03.020135    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:05.020115817 +0000 UTC m=+37.618092155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:03.055698 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.055636    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="784bc2cdfae61c086bd0eee3328edb705f98aebd1b2462df14c8d0492f032c7b" exitCode=0
Jul 29 20:47:03.055698 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.055700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"784bc2cdfae61c086bd0eee3328edb705f98aebd1b2462df14c8d0492f032c7b"}
Jul 29 20:47:03.085753 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.085709    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:03.336676 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.336482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:03.389916 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:03.389883    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:04.060910 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:04.060869    2569 generic.go:356] "Generic (PLEG): container finished" podID="48e16bae-b350-46ad-bd4c-c5c7e288ebf1" containerID="b93bc0764c6bc676394feae71c54f332d2d8b56f8504e6beebeae043ee463041" exitCode=0
Jul 29 20:47:04.060910 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:04.060912    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerDied","Data":"b93bc0764c6bc676394feae71c54f332d2d8b56f8504e6beebeae043ee463041"}
Jul 29 20:47:05.031632 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.031604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:05.031765 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:05.031748    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:05.031821 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:05.031810    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:09.031796012 +0000 UTC m=+41.629772347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:05.064885 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.064853    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sfgfh" event={"ID":"03c2ee14-ab98-46df-96f7-312d4fd0b652","Type":"ContainerStarted","Data":"60439ee079722c71888ae6dc157e5fe5ae942f00dcde5342b1ed062e07f264b6"}
Jul 29 20:47:05.068296 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.068264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fkswq" event={"ID":"48e16bae-b350-46ad-bd4c-c5c7e288ebf1","Type":"ContainerStarted","Data":"9e02851d36175593d1c56fd5295b3a9669d38a628320d30d7e5fa95a073e516c"}
Jul 29 20:47:05.081092 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.080993    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sfgfh" podStartSLOduration=1.8435500139999998 podStartE2EDuration="4.080975433s" totalImagesPullingTime="2.237425419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:05.080708603 +0000 UTC m=+37.678684978" watchObservedRunningTime="2026-07-29 20:47:05.080975433 +0000 UTC m=+37.678951789"
Jul 29 20:47:05.100416 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.100381    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fkswq" podStartSLOduration=10.42250259 podStartE2EDuration="38.100350715s" totalImagesPullingTime="27.677848125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:05.099055144 +0000 UTC m=+37.697031524" watchObservedRunningTime="2026-07-29 20:47:05.100350715 +0000 UTC m=+37.698327098"
Jul 29 20:47:05.219311 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.219288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:05.277210 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:05.277151    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:06.333553 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:06.333495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:06.434674 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:06.434630    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:08.704966 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.704939    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-75r9g"]
Jul 29 20:47:08.733837 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.733803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:08.739797 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.739778    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-75r9g"]
Jul 29 20:47:08.739897 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.739872    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.742335 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.742312    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2kmm5\""
Jul 29 20:47:08.742488 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.742476    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 20:47:08.743318 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.743297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 20:47:08.743380 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.743305    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 20:47:08.743428 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.743321    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 20:47:08.753001 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.752981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e0b4ac78-fa1b-4793-8c29-19b513f368e0-crio-socket\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.753078 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.753025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl5vg\" (UniqueName: \"kubernetes.io/projected/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-api-access-cl5vg\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.753078 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.753061    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.753140 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.753090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0b4ac78-fa1b-4793-8c29-19b513f368e0-data-volume\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.753140 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.753107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.790294 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.790271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:08.854047 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e0b4ac78-fa1b-4793-8c29-19b513f368e0-crio-socket\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854125 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854094    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl5vg\" (UniqueName: \"kubernetes.io/projected/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-api-access-cl5vg\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854180 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854220 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0b4ac78-fa1b-4793-8c29-19b513f368e0-data-volume\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854266 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e0b4ac78-fa1b-4793-8c29-19b513f368e0-crio-socket\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854266 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.854353 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:08.854303    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:47:08.854353 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:08.854323    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:08.854447 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:08.854372    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:09.354353779 +0000 UTC m=+41.952330138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : configmap "kube-rbac-proxy" not found
Jul 29 20:47:08.854447 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:08.854435    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:09.354380668 +0000 UTC m=+41.952357020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:08.854585 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.854558    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0b4ac78-fa1b-4793-8c29-19b513f368e0-data-volume\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:08.871944 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:08.871926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl5vg\" (UniqueName: \"kubernetes.io/projected/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-api-access-cl5vg\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:09.008439 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.008413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:09.055678 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.055652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:09.055793 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.055776    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:09.055857 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.055824    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:17.055809207 +0000 UTC m=+49.653785543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:09.056285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.056259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:09.357889 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.357841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:09.357889 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.357873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:09.358031 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.357960    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:09.358031 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.357965    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:47:09.358031 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.358006    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:10.357995211 +0000 UTC m=+42.955971545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:09.358031 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:09.358018    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:10.35801208 +0000 UTC m=+42.955988414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : configmap "kube-rbac-proxy" not found
Jul 29 20:47:09.760480 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.760457    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:09.825514 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.825489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:09.994668 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:09.994643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.006537 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.006514    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"]
Jul 29 20:47:10.031665 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.031620    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"]
Jul 29 20:47:10.031755 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.031695    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"
Jul 29 20:47:10.034049 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.034024    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 29 20:47:10.035265 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.035247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lfvpx\""
Jul 29 20:47:10.035332 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.035247    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 29 20:47:10.061795 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.061772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8d4k\" (UniqueName: \"kubernetes.io/projected/9655674a-1fb8-427a-b965-fc4c6e7c17eb-kube-api-access-b8d4k\") pod \"migrator-6865c9967-754pb\" (UID: \"9655674a-1fb8-427a-b965-fc4c6e7c17eb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"
Jul 29 20:47:10.089614 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.089593    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.162082 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.162058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8d4k\" (UniqueName: \"kubernetes.io/projected/9655674a-1fb8-427a-b965-fc4c6e7c17eb-kube-api-access-b8d4k\") pod \"migrator-6865c9967-754pb\" (UID: \"9655674a-1fb8-427a-b965-fc4c6e7c17eb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"
Jul 29 20:47:10.169416 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.169389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8d4k\" (UniqueName: \"kubernetes.io/projected/9655674a-1fb8-427a-b965-fc4c6e7c17eb-kube-api-access-b8d4k\") pod \"migrator-6865c9967-754pb\" (UID: \"9655674a-1fb8-427a-b965-fc4c6e7c17eb\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"
Jul 29 20:47:10.190821 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.190798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.241802 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.241784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.341370 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.341323    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"
Jul 29 20:47:10.363653 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.363633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:10.363716 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.363663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:10.363782 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:10.363750    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:47:10.363782 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:10.363769    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:10.363854 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:10.363797    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:12.363784136 +0000 UTC m=+44.961760470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : configmap "kube-rbac-proxy" not found
Jul 29 20:47:10.363854 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:10.363811    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:12.363804547 +0000 UTC m=+44.961780881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:10.467902 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.467873    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.496753 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.496711    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-754pb"]
Jul 29 20:47:10.499341 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:10.499317    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9655674a_1fb8_427a_b965_fc4c6e7c17eb.slice/crio-d6a4f4e80eda53bb624cbf8dc78a14bf66a4f8187a11057327eb99dc79ca842d WatchSource:0}: Error finding container d6a4f4e80eda53bb624cbf8dc78a14bf66a4f8187a11057327eb99dc79ca842d: Status 404 returned error can't find the container with id d6a4f4e80eda53bb624cbf8dc78a14bf66a4f8187a11057327eb99dc79ca842d
Jul 29 20:47:10.566963 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.566938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:10.903419 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:10.903400    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wx7q5_d35b2869-d157-4f59-a05e-3dd14b1ff33e/dns-node-resolver/0.log"
Jul 29 20:47:11.083687 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:11.083652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb" event={"ID":"9655674a-1fb8-427a-b965-fc4c6e7c17eb","Type":"ContainerStarted","Data":"d6a4f4e80eda53bb624cbf8dc78a14bf66a4f8187a11057327eb99dc79ca842d"}
Jul 29 20:47:11.703661 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:11.703635    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j96sc_fd012239-a602-4f38-a26e-ecdaad610b62/node-ca/0.log"
Jul 29 20:47:12.055878 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.055859    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7xz99"]
Jul 29 20:47:12.073940 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.073921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7xz99"]
Jul 29 20:47:12.074039 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.074028    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.076426 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.076411    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 20:47:12.076513 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.076435    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 20:47:12.076513 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.076450    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 20:47:12.077530 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.077512    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 20:47:12.077611 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.077563    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8txrk\""
Jul 29 20:47:12.086779 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.086759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb" event={"ID":"9655674a-1fb8-427a-b965-fc4c6e7c17eb","Type":"ContainerStarted","Data":"3533fce65521838c9fcfe21b210c6c60efcb86b778515d43cab10b1bd689da3b"}
Jul 29 20:47:12.093627 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.093606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:12.148361 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.148342    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:12.175563 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.175542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-cabundle\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.175650 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.175574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-key\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.175718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.175700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhj7z\" (UniqueName: \"kubernetes.io/projected/89e87ca5-ac87-4add-9261-fbe2172f7b78-kube-api-access-dhj7z\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.276311 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.276291    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhj7z\" (UniqueName: \"kubernetes.io/projected/89e87ca5-ac87-4add-9261-fbe2172f7b78-kube-api-access-dhj7z\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.276397 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.276348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-cabundle\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.276397 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.276366    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-key\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.276946 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.276917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-cabundle\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.278344 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.278326    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/89e87ca5-ac87-4add-9261-fbe2172f7b78-signing-key\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.283310 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.283293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhj7z\" (UniqueName: \"kubernetes.io/projected/89e87ca5-ac87-4add-9261-fbe2172f7b78-kube-api-access-dhj7z\") pod \"service-ca-658d7b7c78-7xz99\" (UID: \"89e87ca5-ac87-4add-9261-fbe2172f7b78\") " pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.377564 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.377542    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:12.377643 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.377578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:12.377711 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:12.377653    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 20:47:12.377711 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:12.377660    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:12.377711 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:12.377707    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:16.377691618 +0000 UTC m=+48.975667970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:12.377893 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:12.377720    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:16.37771433 +0000 UTC m=+48.975690664 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : configmap "kube-rbac-proxy" not found
Jul 29 20:47:12.382555 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.382534    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-7xz99"
Jul 29 20:47:12.505888 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.505867    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sfgfh_03c2ee14-ab98-46df-96f7-312d4fd0b652/serve-healthcheck-canary/0.log"
Jul 29 20:47:12.509573 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.509550    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-7xz99"]
Jul 29 20:47:12.513065 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:12.513039    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e87ca5_ac87_4add_9261_fbe2172f7b78.slice/crio-f7de2516dd9bb416188590c895715f42642a1fef82e205ffa234c289dfa475f5 WatchSource:0}: Error finding container f7de2516dd9bb416188590c895715f42642a1fef82e205ffa234c289dfa475f5: Status 404 returned error can't find the container with id f7de2516dd9bb416188590c895715f42642a1fef82e205ffa234c289dfa475f5
Jul 29 20:47:12.945032 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:12.945007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:13.020944 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:13.020916    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:13.090150 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:13.090126    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb" event={"ID":"9655674a-1fb8-427a-b965-fc4c6e7c17eb","Type":"ContainerStarted","Data":"698e64421e894532eea513d2ffaca643934a0444549b85f210d212705fb90254"}
Jul 29 20:47:13.091194 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:13.091173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7xz99" event={"ID":"89e87ca5-ac87-4add-9261-fbe2172f7b78","Type":"ContainerStarted","Data":"f7de2516dd9bb416188590c895715f42642a1fef82e205ffa234c289dfa475f5"}
Jul 29 20:47:13.107660 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:13.107600    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-754pb" podStartSLOduration=2.6478847979999998 podStartE2EDuration="4.107585635s" totalImagesPullingTime="1.459700837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:13.107263431 +0000 UTC m=+45.705239787" watchObservedRunningTime="2026-07-29 20:47:13.107585635 +0000 UTC m=+45.705562021"
Jul 29 20:47:15.343111 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:15.343079    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:15.821115 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:15.821084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:15.879540 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:15.879515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:16.099488 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.099427    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-7xz99" event={"ID":"89e87ca5-ac87-4add-9261-fbe2172f7b78","Type":"ContainerStarted","Data":"47b11c347d2a1e8f02053a12fb31d4a8bf0222387060a69a15e0628b9f6e5cde"}
Jul 29 20:47:16.115142 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.115100    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-7xz99" podStartSLOduration=1.42799214 podStartE2EDuration="4.115086905s" totalImagesPullingTime="2.687094765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:16.114764854 +0000 UTC m=+48.712741202" watchObservedRunningTime="2026-07-29 20:47:16.115086905 +0000 UTC m=+48.713063260"
Jul 29 20:47:16.409583 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.409533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:16.409583 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.409568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:16.409956 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:16.409717    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:16.409956 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:16.409774    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls podName:e0b4ac78-fa1b-4793-8c29-19b513f368e0 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:24.409763198 +0000 UTC m=+57.007739536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-75r9g" (UID: "e0b4ac78-fa1b-4793-8c29-19b513f368e0") : secret "insights-runtime-extractor-tls" not found
Jul 29 20:47:16.410112 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.410090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e0b4ac78-fa1b-4793-8c29-19b513f368e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:16.838849 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.838812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:16.949113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:16.949079    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:17.054834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.054797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:17.114592 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.114514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:17.114754 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:17.114692    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 20:47:17.114820 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:17.114770    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls podName:9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:33.114751341 +0000 UTC m=+65.712727689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls") pod "dns-default-r8v9m" (UID: "9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7") : secret "dns-default-metrics-tls" not found
Jul 29 20:47:17.156446 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.156422    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:17.254507 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.254475    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:17.370904 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.370834    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:17.527885 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:17.527861    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:18.588196 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:18.588166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:18.923888 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:18.923812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:19.122685 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:19.122652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:19.349365 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:19.349336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:19.523251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:19.523222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:20.563593 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:20.563557    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:20.614174 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:20.614122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:20.937249 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:20.937170    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:20.992247 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:20.992222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:21.122855 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:21.122830    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:21.326476 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:21.326451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:21.724854 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:21.724832    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:21.779389 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:21.779334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:22.523752 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:22.523707    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:22.625241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:22.625201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:22.691760 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:22.691722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:22.928102 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:22.928041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:23.130213 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:23.130184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:23.320519 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:23.320494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:23.723798 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:23.723773    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:24.156878 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.156799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:24.325580 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.325550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:24.467509 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.467420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:24.469761 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.469714    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e0b4ac78-fa1b-4793-8c29-19b513f368e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-75r9g\" (UID: \"e0b4ac78-fa1b-4793-8c29-19b513f368e0\") " pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:24.648402 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.648360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-75r9g"
Jul 29 20:47:24.664365 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.664340    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:24.767686 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.767651    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:24.770657 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:24.768542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-75r9g"]
Jul 29 20:47:24.773612 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:24.773589    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0b4ac78_fa1b_4793_8c29_19b513f368e0.slice/crio-120402387ad8c11aa9e68c64b154109c4b583c5078171d56faa573c722825eaf WatchSource:0}: Error finding container 120402387ad8c11aa9e68c64b154109c4b583c5078171d56faa573c722825eaf: Status 404 returned error can't find the container with id 120402387ad8c11aa9e68c64b154109c4b583c5078171d56faa573c722825eaf
Jul 29 20:47:25.121425 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:25.121344    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75r9g" event={"ID":"e0b4ac78-fa1b-4793-8c29-19b513f368e0","Type":"ContainerStarted","Data":"120402387ad8c11aa9e68c64b154109c4b583c5078171d56faa573c722825eaf"}
Jul 29 20:47:25.139525 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:25.139493    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:25.326495 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:25.326450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:25.725660 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:25.725590    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:26.047637 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:26.047555    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sljj6"
Jul 29 20:47:26.325034 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:26.324940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:26.926452 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:26.926417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:27.130227 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:27.130187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:27.327549 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:27.327512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:27.548682 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:27.548642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.032388 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.032358    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"]
Jul 29 20:47:30.034446 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.034432    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.037007 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.036982    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 20:47:30.038002 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.037980    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-fdbnf\""
Jul 29 20:47:30.038091 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.038003    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 20:47:30.038091 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.038012    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 20:47:30.038091 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.038016    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 20:47:30.046906 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.046885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"]
Jul 29 20:47:30.115286 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.115216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ef4e59b-9c8f-444a-8945-6243c410f3a9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.115286 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.115267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx5g8\" (UniqueName: \"kubernetes.io/projected/6ef4e59b-9c8f-444a-8945-6243c410f3a9-kube-api-access-cx5g8\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.139089 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.139047    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75r9g" event={"ID":"e0b4ac78-fa1b-4793-8c29-19b513f368e0","Type":"ContainerStarted","Data":"b6a29516377543360e22214c28e071ce10c9abbecbaf57d0d8c0512ca16d07c7"}
Jul 29 20:47:30.146694 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.146671    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-789894bd89-pzz4w"]
Jul 29 20:47:30.150304 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.150246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.150304 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.150283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.153075 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.153053    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wf87m\""
Jul 29 20:47:30.153403 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.153383    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 20:47:30.154252 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.153958    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 20:47:30.155031 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.154839    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 20:47:30.160662 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.160638    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 20:47:30.166759 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.166690    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789894bd89-pzz4w"]
Jul 29 20:47:30.216013 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.215978    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-bound-sa-token\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216176 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ef4e59b-9c8f-444a-8945-6243c410f3a9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.216176 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216080    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-registry-certificates\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216176 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-trusted-ca\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216176 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216142    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx5g8\" (UniqueName: \"kubernetes.io/projected/6ef4e59b-9c8f-444a-8945-6243c410f3a9-kube-api-access-cx5g8\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.216441 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bee974dc-0569-424d-bfda-cfa61af808d0-ca-trust-extracted\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216441 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216240    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-registry-tls\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216441 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216267    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-image-registry-private-configuration\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216441 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-installation-pull-secrets\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.216441 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.216316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2zbg\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-kube-api-access-l2zbg\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.219119 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.219095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6ef4e59b-9c8f-444a-8945-6243c410f3a9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.225843 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.225820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx5g8\" (UniqueName: \"kubernetes.io/projected/6ef4e59b-9c8f-444a-8945-6243c410f3a9-kube-api-access-cx5g8\") pod \"managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh\" (UID: \"6ef4e59b-9c8f-444a-8945-6243c410f3a9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.237540 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.237513    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.317137 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-registry-tls\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317137 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317102    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-image-registry-private-configuration\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317137 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-installation-pull-secrets\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317154    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2zbg\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-kube-api-access-l2zbg\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-bound-sa-token\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-registry-certificates\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.317404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-trusted-ca\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.320003 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.317843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bee974dc-0569-424d-bfda-cfa61af808d0-ca-trust-extracted\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.320003 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.318435    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bee974dc-0569-424d-bfda-cfa61af808d0-ca-trust-extracted\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.320003 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.319274    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-trusted-ca\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.322547 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.321452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bee974dc-0569-424d-bfda-cfa61af808d0-registry-certificates\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.326511 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.326452    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-registry-tls\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.332008 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.331982    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-installation-pull-secrets\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.332646 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.332621    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bee974dc-0569-424d-bfda-cfa61af808d0-image-registry-private-configuration\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.334057 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.334034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2zbg\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-kube-api-access-l2zbg\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.334953 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.334911    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bee974dc-0569-424d-bfda-cfa61af808d0-bound-sa-token\") pod \"image-registry-789894bd89-pzz4w\" (UID: \"bee974dc-0569-424d-bfda-cfa61af808d0\") " pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.356283 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.356237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"
Jul 29 20:47:30.409562 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.409508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.467178 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.467043    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:30.487242 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.487176    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.504698 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.504642    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh"]
Jul 29 20:47:30.510364 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:30.510335    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ef4e59b_9c8f_444a_8945_6243c410f3a9.slice/crio-2f13653d68622c97038866acbcfe522318ba037b206463b9bcd01067258f9934 WatchSource:0}: Error finding container 2f13653d68622c97038866acbcfe522318ba037b206463b9bcd01067258f9934: Status 404 returned error can't find the container with id 2f13653d68622c97038866acbcfe522318ba037b206463b9bcd01067258f9934
Jul 29 20:47:30.620009 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.619862    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-789894bd89-pzz4w"]
Jul 29 20:47:30.624344 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:30.624315    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee974dc_0569_424d_bfda_cfa61af808d0.slice/crio-b127449e1d0c9532ab3325546b05e58065f8c0021c0fd8724d1306916367e378 WatchSource:0}: Error finding container b127449e1d0c9532ab3325546b05e58065f8c0021c0fd8724d1306916367e378: Status 404 returned error can't find the container with id b127449e1d0c9532ab3325546b05e58065f8c0021c0fd8724d1306916367e378
Jul 29 20:47:30.812027 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.811977    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:30.954004 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:30.953901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:31.144298 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:31.144258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh" event={"ID":"6ef4e59b-9c8f-444a-8945-6243c410f3a9","Type":"ContainerStarted","Data":"2f13653d68622c97038866acbcfe522318ba037b206463b9bcd01067258f9934"}
Jul 29 20:47:31.146362 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:31.146333    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789894bd89-pzz4w" event={"ID":"bee974dc-0569-424d-bfda-cfa61af808d0","Type":"ContainerStarted","Data":"a45b8b48ef049a4fa93b322973ea33680efdddbb6f746864f67bb56bef9dd9a7"}
Jul 29 20:47:31.146496 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:31.146384    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-789894bd89-pzz4w" event={"ID":"bee974dc-0569-424d-bfda-cfa61af808d0","Type":"ContainerStarted","Data":"b127449e1d0c9532ab3325546b05e58065f8c0021c0fd8724d1306916367e378"}
Jul 29 20:47:31.146633 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:31.146602    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:32.391537 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:32.391495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:32.497584 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:32.497546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:33.143208 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.143165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:33.145657 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.145611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7-metrics-tls\") pod \"dns-default-r8v9m\" (UID: \"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7\") " pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:33.416039 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.415970    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-79k6t\""
Jul 29 20:47:33.424538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.424508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:33.545952 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.545894    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:33.548757 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.548714    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 20:47:33.559202 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.559166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad1052f7-0f92-4807-a179-014bf646b54f-metrics-certs\") pod \"network-metrics-daemon-jj8sp\" (UID: \"ad1052f7-0f92-4807-a179-014bf646b54f\") " pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:33.646581 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.646539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:33.649521 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.649497    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 20:47:33.660197 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.660167    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2a65e073-18d7-43d6-a600-eab03235eff9-original-pull-secret\") pod \"global-pull-secret-syncer-m96cs\" (UID: \"2a65e073-18d7-43d6-a600-eab03235eff9\") " pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:33.746982 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.746944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:33.749724 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.749702    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 20:47:33.760333 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.760311    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 20:47:33.770938 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.770912    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwsqt\" (UniqueName: \"kubernetes.io/projected/9e1927f3-8f82-4454-a39e-dc8f4e9719d9-kube-api-access-zwsqt\") pod \"network-check-target-lpmln\" (UID: \"9e1927f3-8f82-4454-a39e-dc8f4e9719d9\") " pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:33.802731 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.802700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m96cs"
Jul 29 20:47:33.811408 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.811385    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lzlv4\""
Jul 29 20:47:33.819512 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:33.819494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jj8sp"
Jul 29 20:47:34.005684 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:34.005615    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zcmz5\""
Jul 29 20:47:34.013706 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:34.013682    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:35.210951 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:35.210875    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-789894bd89-pzz4w" podStartSLOduration=5.210854126 podStartE2EDuration="5.210854126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:31.16654031 +0000 UTC m=+63.764516668" watchObservedRunningTime="2026-07-29 20:47:35.210854126 +0000 UTC m=+67.808830484"
Jul 29 20:47:35.211613 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:35.211595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jj8sp"]
Jul 29 20:47:35.220651 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:35.220610    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad1052f7_0f92_4807_a179_014bf646b54f.slice/crio-0677b3146eed0f7e6bd1ac1f60ce3b46429560c6dbb7fe76bacb4f195d8a72f2 WatchSource:0}: Error finding container 0677b3146eed0f7e6bd1ac1f60ce3b46429560c6dbb7fe76bacb4f195d8a72f2: Status 404 returned error can't find the container with id 0677b3146eed0f7e6bd1ac1f60ce3b46429560c6dbb7fe76bacb4f195d8a72f2
Jul 29 20:47:35.426997 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:35.426921    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m96cs"]
Jul 29 20:47:35.427865 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:35.427802    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lpmln"]
Jul 29 20:47:35.429271 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:35.429235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r8v9m"]
Jul 29 20:47:35.430073 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:35.429973    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1927f3_8f82_4454_a39e_dc8f4e9719d9.slice/crio-bd6202a7f763160b8be44d5d7afd1bbdce08b52f3ea350e027b4ca6e6ea62249 WatchSource:0}: Error finding container bd6202a7f763160b8be44d5d7afd1bbdce08b52f3ea350e027b4ca6e6ea62249: Status 404 returned error can't find the container with id bd6202a7f763160b8be44d5d7afd1bbdce08b52f3ea350e027b4ca6e6ea62249
Jul 29 20:47:35.430579 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:35.430554    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a65e073_18d7_43d6_a600_eab03235eff9.slice/crio-87ea4fd8ef4034a21f8e814936ecf682728f1e46d9175c0c6c6cbf78fe1faa75 WatchSource:0}: Error finding container 87ea4fd8ef4034a21f8e814936ecf682728f1e46d9175c0c6c6cbf78fe1faa75: Status 404 returned error can't find the container with id 87ea4fd8ef4034a21f8e814936ecf682728f1e46d9175c0c6c6cbf78fe1faa75
Jul 29 20:47:35.431843 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:35.431717    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d34bdd2_54b1_4b36_a707_a0b72a9e3ae7.slice/crio-1a190bed336ca819507a16745792eb5265c48ef8bd8ec1b61e38b34758e95a1b WatchSource:0}: Error finding container 1a190bed336ca819507a16745792eb5265c48ef8bd8ec1b61e38b34758e95a1b: Status 404 returned error can't find the container with id 1a190bed336ca819507a16745792eb5265c48ef8bd8ec1b61e38b34758e95a1b
Jul 29 20:47:36.168884 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.168844    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75r9g" event={"ID":"e0b4ac78-fa1b-4793-8c29-19b513f368e0","Type":"ContainerStarted","Data":"625616da584653ac153f8beee5c891f62482291937475f83388068ff170f33f4"}
Jul 29 20:47:36.171151 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.171068    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jj8sp" event={"ID":"ad1052f7-0f92-4807-a179-014bf646b54f","Type":"ContainerStarted","Data":"0677b3146eed0f7e6bd1ac1f60ce3b46429560c6dbb7fe76bacb4f195d8a72f2"}
Jul 29 20:47:36.173072 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.172998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8v9m" event={"ID":"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7","Type":"ContainerStarted","Data":"1a190bed336ca819507a16745792eb5265c48ef8bd8ec1b61e38b34758e95a1b"}
Jul 29 20:47:36.175793 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.175063    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh" event={"ID":"6ef4e59b-9c8f-444a-8945-6243c410f3a9","Type":"ContainerStarted","Data":"507debf154c88765174d8e9aaa5d1a5c23b0f9c984a5ddca62427786dd8d4675"}
Jul 29 20:47:36.178046 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.178019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpmln" event={"ID":"9e1927f3-8f82-4454-a39e-dc8f4e9719d9","Type":"ContainerStarted","Data":"bd6202a7f763160b8be44d5d7afd1bbdce08b52f3ea350e027b4ca6e6ea62249"}
Jul 29 20:47:36.185821 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.185798    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m96cs" event={"ID":"2a65e073-18d7-43d6-a600-eab03235eff9","Type":"ContainerStarted","Data":"87ea4fd8ef4034a21f8e814936ecf682728f1e46d9175c0c6c6cbf78fe1faa75"}
Jul 29 20:47:36.190863 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:36.190817    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69b9bd8b6-fbfbh" podStartSLOduration=1.6564307679999999 podStartE2EDuration="6.190799835s" totalImagesPullingTime="4.534369067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:36.189643423 +0000 UTC m=+68.787619779" watchObservedRunningTime="2026-07-29 20:47:36.190799835 +0000 UTC m=+68.788776194"
Jul 29 20:47:37.193625 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:37.193559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jj8sp" event={"ID":"ad1052f7-0f92-4807-a179-014bf646b54f","Type":"ContainerStarted","Data":"977b65f799933c10bc389ab5897133c0ac239ee1ce7908defb7062b08a0d2c72"}
Jul 29 20:47:37.194499 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:37.193794    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jj8sp" event={"ID":"ad1052f7-0f92-4807-a179-014bf646b54f","Type":"ContainerStarted","Data":"c734f10c8e4963e81418e1db736995aff16182e9ca8234f6de0be7d889846651"}
Jul 29 20:47:37.211100 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:37.211011    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jj8sp" podStartSLOduration=69.004641594 podStartE2EDuration="1m10.210992646s" totalImagesPullingTime="1.206351052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:46:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:37.208920272 +0000 UTC m=+69.806896642" watchObservedRunningTime="2026-07-29 20:47:37.210992646 +0000 UTC m=+69.808969005"
Jul 29 20:47:37.302153 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:37.302088    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:37.359113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:37.359035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:38.199489 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:38.199388    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8v9m" event={"ID":"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7","Type":"ContainerStarted","Data":"39db6590e35716fb38e8c50ad1aec62901d5de4eee64a150317ad8fa98c7c795"}
Jul 29 20:47:39.269515 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.269485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.372994 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.372937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.437513 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.437479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.441398 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.441373    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"]
Jul 29 20:47:39.451513 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.451488    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:39.454044 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.454024    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s64w4\""
Jul 29 20:47:39.454171 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.454029    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 20:47:39.458442 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.458016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"]
Jul 29 20:47:39.586489 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.586409    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.591424 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.591397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e18423f2-a537-409e-9137-983d1c2e237f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6wmg8\" (UID: \"e18423f2-a537-409e-9137-983d1c2e237f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:39.692100 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.692067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e18423f2-a537-409e-9137-983d1c2e237f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6wmg8\" (UID: \"e18423f2-a537-409e-9137-983d1c2e237f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:39.694411 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.694388    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/e18423f2-a537-409e-9137-983d1c2e237f-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6wmg8\" (UID: \"e18423f2-a537-409e-9137-983d1c2e237f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:39.764523 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.764497    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:39.803478 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.803450    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-5j86x"]
Jul 29 20:47:39.805409 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.805390    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:47:39.808294 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.808271    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 20:47:39.808410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.808347    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-pg8fv\""
Jul 29 20:47:39.808590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.808575    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 20:47:39.817523 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.817503    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5j86x"]
Jul 29 20:47:39.826315 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.826288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.892913 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.892836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brst9\" (UniqueName: \"kubernetes.io/projected/cf091058-cc54-4fdf-92fc-601a36e78ea9-kube-api-access-brst9\") pod \"downloads-7c97dc4555-5j86x\" (UID: \"cf091058-cc54-4fdf-92fc-601a36e78ea9\") " pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:47:39.928241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.928212    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:39.993520 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:39.993492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brst9\" (UniqueName: \"kubernetes.io/projected/cf091058-cc54-4fdf-92fc-601a36e78ea9-kube-api-access-brst9\") pod \"downloads-7c97dc4555-5j86x\" (UID: \"cf091058-cc54-4fdf-92fc-601a36e78ea9\") " pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:47:40.002087 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.002059    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brst9\" (UniqueName: \"kubernetes.io/projected/cf091058-cc54-4fdf-92fc-601a36e78ea9-kube-api-access-brst9\") pod \"downloads-7c97dc4555-5j86x\" (UID: \"cf091058-cc54-4fdf-92fc-601a36e78ea9\") " pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:47:40.115958 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.115925    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:47:40.189450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.189350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.292049 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.292012    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.368582 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.368546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.477591 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.477564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.530666 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.530640    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.922646 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.922560    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:40.984547 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:40.984517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:41.694457 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:41.694386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:41.895936 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:41.895875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:42.092029 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.091999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:42.209272 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.209233    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-5j86x"]
Jul 29 20:47:42.213948 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:42.213922    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf091058_cc54_4fdf_92fc_601a36e78ea9.slice/crio-a9bdda692c446e1eb8988ec0e0484d8c35b8a569ec0c57c9a31f607ca641bded WatchSource:0}: Error finding container a9bdda692c446e1eb8988ec0e0484d8c35b8a569ec0c57c9a31f607ca641bded: Status 404 returned error can't find the container with id a9bdda692c446e1eb8988ec0e0484d8c35b8a569ec0c57c9a31f607ca641bded
Jul 29 20:47:42.221235 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.221137    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"]
Jul 29 20:47:42.222196 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.221828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m96cs" event={"ID":"2a65e073-18d7-43d6-a600-eab03235eff9","Type":"ContainerStarted","Data":"d8da9a718b814855b5c6499c2442a230e6ab223fd41bd3b39152740b58c1d321"}
Jul 29 20:47:42.223072 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:42.223048    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18423f2_a537_409e_9137_983d1c2e237f.slice/crio-a0d90599e01fb3dc77e294835db4133aa00f05889e80eea3983748db5c9fb34d WatchSource:0}: Error finding container a0d90599e01fb3dc77e294835db4133aa00f05889e80eea3983748db5c9fb34d: Status 404 returned error can't find the container with id a0d90599e01fb3dc77e294835db4133aa00f05889e80eea3983748db5c9fb34d
Jul 29 20:47:42.224666 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.224641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-75r9g" event={"ID":"e0b4ac78-fa1b-4793-8c29-19b513f368e0","Type":"ContainerStarted","Data":"ea77db6e47cf5ea3acaecd7b6f1daac565726be916c6d5b3d02f47e92655f7db"}
Jul 29 20:47:42.227344 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.226928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r8v9m" event={"ID":"9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7","Type":"ContainerStarted","Data":"fe45af91fe59fee3318d9aff0eb3f2ed2d58befaf5ee657d3ddf484014934241"}
Jul 29 20:47:42.227434 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.227394    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:42.228730 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.228708    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lpmln" event={"ID":"9e1927f3-8f82-4454-a39e-dc8f4e9719d9","Type":"ContainerStarted","Data":"875c2e06d7a7d223f6460029c46041f715683eefd809d832e0aaa37601116854"}
Jul 29 20:47:42.228868 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.228854    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:47:42.236038 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.236001    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-m96cs" podStartSLOduration=67.623104398 podStartE2EDuration="1m14.235989874s" totalImagesPullingTime="6.612885476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:42.2355365 +0000 UTC m=+74.833512872" watchObservedRunningTime="2026-07-29 20:47:42.235989874 +0000 UTC m=+74.833966230"
Jul 29 20:47:42.250443 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.250360    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lpmln" podStartSLOduration=67.633215671 podStartE2EDuration="1m14.250344256s" totalImagesPullingTime="6.617128585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:42.249149839 +0000 UTC m=+74.847126195" watchObservedRunningTime="2026-07-29 20:47:42.250344256 +0000 UTC m=+74.848320613"
Jul 29 20:47:42.265498 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.265453    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-75r9g" podStartSLOduration=17.219269758 podStartE2EDuration="34.265436775s" totalImagesPullingTime="17.046167017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:42.265340391 +0000 UTC m=+74.863316769" watchObservedRunningTime="2026-07-29 20:47:42.265436775 +0000 UTC m=+74.863413135"
Jul 29 20:47:42.280835 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:42.280796    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r8v9m" podStartSLOduration=38.894698989 podStartE2EDuration="41.280785656s" totalImagesPullingTime="2.386086667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:42.280059033 +0000 UTC m=+74.878035397" watchObservedRunningTime="2026-07-29 20:47:42.280785656 +0000 UTC m=+74.878762024"
Jul 29 20:47:43.117018 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:43.116954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:43.167645 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:43.167608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:43.234479 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:43.234441    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5j86x" event={"ID":"cf091058-cc54-4fdf-92fc-601a36e78ea9","Type":"ContainerStarted","Data":"a9bdda692c446e1eb8988ec0e0484d8c35b8a569ec0c57c9a31f607ca641bded"}
Jul 29 20:47:43.239167 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:43.237825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8" event={"ID":"e18423f2-a537-409e-9137-983d1c2e237f","Type":"ContainerStarted","Data":"a0d90599e01fb3dc77e294835db4133aa00f05889e80eea3983748db5c9fb34d"}
Jul 29 20:47:44.242798 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:44.242750    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8" event={"ID":"e18423f2-a537-409e-9137-983d1c2e237f","Type":"ContainerStarted","Data":"6753a8fbae126250396a6ad8b1c93ccd7f9f6caf8d8fff4f44c82057bbf6fb7f"}
Jul 29 20:47:44.258281 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:44.258089    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8" podStartSLOduration=3.975311951 podStartE2EDuration="5.25807372s" totalImagesPullingTime="1.282761769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:44.257758016 +0000 UTC m=+76.855734376" watchObservedRunningTime="2026-07-29 20:47:44.25807372 +0000 UTC m=+76.856050078"
Jul 29 20:47:45.246060 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.246031    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:45.251020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.250996    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6wmg8"
Jul 29 20:47:45.465651 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.465605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:45.501657 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.501586    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"]
Jul 29 20:47:45.503799 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.503780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.507149 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507114    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 20:47:45.507297 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507179    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 20:47:45.507297 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507241    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h9k6r\""
Jul 29 20:47:45.507434 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507329    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 20:47:45.507434 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507179    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 20:47:45.507554 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.507498    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 20:47:45.511850 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.511827    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"]
Jul 29 20:47:45.529941 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.529904    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:45.639341 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.639313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.639440 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.639360    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fppmp\" (UniqueName: \"kubernetes.io/projected/d7714ff7-df3e-4ff7-b718-13f65bb54017-kube-api-access-fppmp\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.639440 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.639416    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.639530 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.639447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7714ff7-df3e-4ff7-b718-13f65bb54017-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.740479 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.740454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.740581 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.740499    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7714ff7-df3e-4ff7-b718-13f65bb54017-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.740581 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.740552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.740692 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:45.740588    2569 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 29 20:47:45.740692 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.740590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fppmp\" (UniqueName: \"kubernetes.io/projected/d7714ff7-df3e-4ff7-b718-13f65bb54017-kube-api-access-fppmp\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.740692 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:45.740658    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls podName:d7714ff7-df3e-4ff7-b718-13f65bb54017 nodeName:}" failed. No retries permitted until 2026-07-29 20:47:46.240641178 +0000 UTC m=+78.838617514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-4v7vr" (UID: "d7714ff7-df3e-4ff7-b718-13f65bb54017") : secret "prometheus-operator-tls" not found
Jul 29 20:47:45.741313 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.741286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7714ff7-df3e-4ff7-b718-13f65bb54017-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.742850 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.742824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.749094 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.749067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fppmp\" (UniqueName: \"kubernetes.io/projected/d7714ff7-df3e-4ff7-b718-13f65bb54017-kube-api-access-fppmp\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:45.814932 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.814875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:45.915466 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.915441    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:45.966942 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:45.966919    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:46.245059 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:46.245020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:46.247611 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:46.247576    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7714ff7-df3e-4ff7-b718-13f65bb54017-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-4v7vr\" (UID: \"d7714ff7-df3e-4ff7-b718-13f65bb54017\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:46.415193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:46.415166    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"
Jul 29 20:47:46.551580 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:46.551469    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-4v7vr"]
Jul 29 20:47:46.554442 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:46.554413    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7714ff7_df3e_4ff7_b718_13f65bb54017.slice/crio-cde505c675bd3531e94c042a2ab5914f7f358c1361771204e42cd84b43c4e383 WatchSource:0}: Error finding container cde505c675bd3531e94c042a2ab5914f7f358c1361771204e42cd84b43c4e383: Status 404 returned error can't find the container with id cde505c675bd3531e94c042a2ab5914f7f358c1361771204e42cd84b43c4e383
Jul 29 20:47:47.245012 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:47.244983    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r8v9m"
Jul 29 20:47:47.254789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:47.254759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr" event={"ID":"d7714ff7-df3e-4ff7-b718-13f65bb54017","Type":"ContainerStarted","Data":"cde505c675bd3531e94c042a2ab5914f7f358c1361771204e42cd84b43c4e383"}
Jul 29 20:47:48.502159 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:48.502049    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:49.264942 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:49.264903    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr" event={"ID":"d7714ff7-df3e-4ff7-b718-13f65bb54017","Type":"ContainerStarted","Data":"a0db990756a01740e351bc1be8946ceff80bab9ef06e907ca09ee697c3901230"}
Jul 29 20:47:49.265222 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:49.265005    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr" event={"ID":"d7714ff7-df3e-4ff7-b718-13f65bb54017","Type":"ContainerStarted","Data":"bb80cae1da1a99d9391dfeffdc15f3d04b1d7b22df47e984ffe599694c292766"}
Jul 29 20:47:49.284241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:49.283873    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-4v7vr" podStartSLOduration=2.599563033 podStartE2EDuration="4.283855571s" totalImagesPullingTime="1.684292538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:47:49.283670406 +0000 UTC m=+81.881646763" watchObservedRunningTime="2026-07-29 20:47:49.283855571 +0000 UTC m=+81.881831931"
Jul 29 20:47:50.472263 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.472143    2569 patch_prober.go:28] interesting pod/image-registry-789894bd89-pzz4w container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 20:47:50.472263 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.472211    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-789894bd89-pzz4w" podUID="bee974dc-0569-424d-bfda-cfa61af808d0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 20:47:50.588855 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.588822    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:50.789162 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.789118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:50.846845 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.846816    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nxs7w"]
Jul 29 20:47:50.850007 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.849768    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.852305 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.852279    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 20:47:50.852689 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.852653    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 20:47:50.854662 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.854186    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xmffz\""
Jul 29 20:47:50.854662 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.854409    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 20:47:50.879764 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.879688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:50.947718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.947691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:50.982146 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982122    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-sys\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982255 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-textfile\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982255 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982198    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982255 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq9hk\" (UniqueName: \"kubernetes.io/projected/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-kube-api-access-sq9hk\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982277    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-wtmp\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-metrics-client-ca\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982328    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-root\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982353    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:50.982410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:50.982380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.052832 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.052619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq9hk\" (UniqueName: \"kubernetes.io/projected/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-kube-api-access-sq9hk\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-wtmp\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-metrics-client-ca\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-root\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083164    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-sys\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-textfile\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.083837    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-accelerators-collector-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.084590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.084211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-wtmp\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.085233 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.084641    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-metrics-client-ca\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.085233 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.084695    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-root\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.085233 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:51.084792    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 20:47:51.085233 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:47:51.084852    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls podName:bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf nodeName:}" failed. No retries permitted until 2026-07-29 20:47:51.584834905 +0000 UTC m=+84.182811247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls") pod "node-exporter-nxs7w" (UID: "bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf") : secret "node-exporter-tls" not found
Jul 29 20:47:51.085686 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.085588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-textfile\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.085686 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.085652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-sys\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.096143 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.096089    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.110872 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.110827    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq9hk\" (UniqueName: \"kubernetes.io/projected/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-kube-api-access-sq9hk\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.151413 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.151386    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.250250 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.250214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.348428 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.348356    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.449116 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.449076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.588328 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.588290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.590926 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.590901    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf-node-exporter-tls\") pod \"node-exporter-nxs7w\" (UID: \"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf\") " pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.761680 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.761651    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nxs7w"
Jul 29 20:47:51.772752 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:47:51.772702    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb9c0cfc_21b1_4656_9b95_f6011c5e4dbf.slice/crio-7c7ec5535463a401954953416efa0238bd6ef1b7dd32503a93e65fe797108c05 WatchSource:0}: Error finding container 7c7ec5535463a401954953416efa0238bd6ef1b7dd32503a93e65fe797108c05: Status 404 returned error can't find the container with id 7c7ec5535463a401954953416efa0238bd6ef1b7dd32503a93e65fe797108c05
Jul 29 20:47:51.816096 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.816063    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:51.893310 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:51.893288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.017187 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.017100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.119274 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.119240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.156396 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.156278    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-789894bd89-pzz4w"
Jul 29 20:47:52.277163 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.277090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxs7w" event={"ID":"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf","Type":"ContainerStarted","Data":"7c7ec5535463a401954953416efa0238bd6ef1b7dd32503a93e65fe797108c05"}
Jul 29 20:47:52.308216 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.308188    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.381280 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.381247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.752222 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.752189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.824560 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.824525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:52.826217 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.826188    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-97c7954bb-vkjvs"]
Jul 29 20:47:52.828912 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.828890    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.831402 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.831376    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 20:47:52.831523 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.831430    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-df8ovrnctn6et\""
Jul 29 20:47:52.832513 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.832476    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 20:47:52.832612 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.832546    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lr5md\""
Jul 29 20:47:52.832612 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.832593    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 20:47:52.832782 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.832488    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 20:47:52.832782 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.832642    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 20:47:52.843610 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.843536    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-97c7954bb-vkjvs"]
Jul 29 20:47:52.999125 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84266f12-a4b3-4d83-8528-4f101873b7ec-metrics-client-ca\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999424 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999424 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999307    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999424 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999373    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-grpc-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:52.999424 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:52.999413    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjc9l\" (UniqueName: \"kubernetes.io/projected/84266f12-a4b3-4d83-8528-4f101873b7ec-kube-api-access-mjc9l\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100320 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100367    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100396    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-grpc-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100640 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100451    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjc9l\" (UniqueName: \"kubernetes.io/projected/84266f12-a4b3-4d83-8528-4f101873b7ec-kube-api-access-mjc9l\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100640 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.100640 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.100516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84266f12-a4b3-4d83-8528-4f101873b7ec-metrics-client-ca\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.101250 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.101226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84266f12-a4b3-4d83-8528-4f101873b7ec-metrics-client-ca\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103122 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103758 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103758 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103905 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103905 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103797    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.103905 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.103837    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84266f12-a4b3-4d83-8528-4f101873b7ec-secret-grpc-tls\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.110145 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.110113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjc9l\" (UniqueName: \"kubernetes.io/projected/84266f12-a4b3-4d83-8528-4f101873b7ec-kube-api-access-mjc9l\") pod \"thanos-querier-97c7954bb-vkjvs\" (UID: \"84266f12-a4b3-4d83-8528-4f101873b7ec\") " pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.140302 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.140281    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:47:53.283915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.283825    2569 generic.go:356] "Generic (PLEG): container finished" podID="bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf" containerID="1242c540d0f9d5bf74409e3b206d042361fe424b466cb8d854bf435cce602b68" exitCode=0
Jul 29 20:47:53.284051 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.283909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxs7w" event={"ID":"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf","Type":"ContainerDied","Data":"1242c540d0f9d5bf74409e3b206d042361fe424b466cb8d854bf435cce602b68"}
Jul 29 20:47:53.287435 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.287401    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-97c7954bb-vkjvs"]
Jul 29 20:47:53.772525 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.772491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:53.874052 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:53.874022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:54.218219 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:54.218132    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:54.896113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:54.896081    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.319169 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.319126    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.391137 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.391103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.611667 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.611595    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.622061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.622035    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"]
Jul 29 20:47:55.627782 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.627725    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.630353 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.630057    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 20:47:55.631166 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.630995    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hbpmm\""
Jul 29 20:47:55.632003 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.631983    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"]
Jul 29 20:47:55.711112 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.711080    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.721865 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.721836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5pm\" (UniqueName: \"kubernetes.io/projected/251fbdae-e030-4ded-bda7-bb7e2dd09689-kube-api-access-dk5pm\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.721986 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.721893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/251fbdae-e030-4ded-bda7-bb7e2dd09689-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.809970 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.809928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:55.823073 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.823039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5pm\" (UniqueName: \"kubernetes.io/projected/251fbdae-e030-4ded-bda7-bb7e2dd09689-kube-api-access-dk5pm\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.823217 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.823125    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/251fbdae-e030-4ded-bda7-bb7e2dd09689-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.825848 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.825819    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/251fbdae-e030-4ded-bda7-bb7e2dd09689-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.832145 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.831263    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5pm\" (UniqueName: \"kubernetes.io/projected/251fbdae-e030-4ded-bda7-bb7e2dd09689-kube-api-access-dk5pm\") pod \"monitoring-plugin-68b7fc9574-kfcjl\" (UID: \"251fbdae-e030-4ded-bda7-bb7e2dd09689\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:55.944423 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:55.944346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:47:56.365546 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:56.365510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:56.469554 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:56.469493    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:57.049249 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.049103    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:47:57.050544 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.050508    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:47:57.105912 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.105816    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:47:57.106071 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.105992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.109538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109348    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 20:47:57.109538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109349    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 20:47:57.109538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109349    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 20:47:57.109538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109353    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 20:47:57.109538 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109462    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 20:47:57.109923 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.109790    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 20:47:57.110053 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110035    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 20:47:57.110111 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110063    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v6dgd\""
Jul 29 20:47:57.110111 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110074    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 20:47:57.110882 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110292    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fb3ih3dqf31v9\""
Jul 29 20:47:57.110882 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110325    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 20:47:57.110882 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110430    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 20:47:57.110882 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110622    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 20:47:57.110882 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110714    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 20:47:57.111238 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.110925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 20:47:57.114707 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.114275    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 20:47:57.117446 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.117418    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 20:47:57.237096 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237058    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237096 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237345 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237128    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237345 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237150    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237773 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237729    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237847    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgmw7\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237973 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237893    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237973 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237934    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.237973 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237958    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.237980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238035    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238058    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238187 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238187 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238187 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238167    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238277 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238277 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238277 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.238277 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.238257    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339434 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339434 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgmw7\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339638 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339638 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339638 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339638 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.339874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.339977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340011    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340063    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340080    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340110 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340445    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.340556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.340497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.341076 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.341034    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.344184 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.344098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.344363 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.344343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.344935 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.344913    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.345043 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.344932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.346560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.346573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347250    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347392    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.347893 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.347645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.348408 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.348363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.348601 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.348577    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.350079 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.350061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgmw7\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7\") pod \"prometheus-k8s-0\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:47:57.420504 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:47:57.420448    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:48:00.679174 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:48:00.679139    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84266f12_a4b3_4d83_8528_4f101873b7ec.slice/crio-6aeaec81d9b137708990c67312ef7f3c8308af19b29e596bde8c49ad21359a19 WatchSource:0}: Error finding container 6aeaec81d9b137708990c67312ef7f3c8308af19b29e596bde8c49ad21359a19: Status 404 returned error can't find the container with id 6aeaec81d9b137708990c67312ef7f3c8308af19b29e596bde8c49ad21359a19
Jul 29 20:48:00.813067 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:00.813036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:00.826826 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:00.826803    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:48:00.828645 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:48:00.828624    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff26dae_8b90_4475_8641_ef5404631c53.slice/crio-0387e273941fc09713a648db57c4fbf7abfc4c6436f2f85c957c2cb9f5a7e87d WatchSource:0}: Error finding container 0387e273941fc09713a648db57c4fbf7abfc4c6436f2f85c957c2cb9f5a7e87d: Status 404 returned error can't find the container with id 0387e273941fc09713a648db57c4fbf7abfc4c6436f2f85c957c2cb9f5a7e87d
Jul 29 20:48:00.835973 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:00.835914    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"]
Jul 29 20:48:00.838350 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:48:00.838328    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251fbdae_e030_4ded_bda7_bb7e2dd09689.slice/crio-4c4b97b0efefe54cf803d053a928c197863447efaa0af63abcff486609258083 WatchSource:0}: Error finding container 4c4b97b0efefe54cf803d053a928c197863447efaa0af63abcff486609258083: Status 404 returned error can't find the container with id 4c4b97b0efefe54cf803d053a928c197863447efaa0af63abcff486609258083
Jul 29 20:48:00.913475 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:00.913448    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:01.116958 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.116933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:01.218680 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.218638    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:01.325220 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.319784    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"0387e273941fc09713a648db57c4fbf7abfc4c6436f2f85c957c2cb9f5a7e87d"}
Jul 29 20:48:01.328930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.328610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxs7w" event={"ID":"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf","Type":"ContainerStarted","Data":"e8fd15f1e3ead3d35a59654ab6cfbc36e23112d292f981bb865d9cf783625cdc"}
Jul 29 20:48:01.328930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.328669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nxs7w" event={"ID":"bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf","Type":"ContainerStarted","Data":"dbf63f13ab99e1ebca47ae4cbe835a9afafe0bde95bc895e528f3a02178353e9"}
Jul 29 20:48:01.331099 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.331068    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-5j86x" event={"ID":"cf091058-cc54-4fdf-92fc-601a36e78ea9","Type":"ContainerStarted","Data":"0054fabb92927bef9898805cdaf8c99873b47acb122d9a4065a12299e5fe8de6"}
Jul 29 20:48:01.331883 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.331813    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:48:01.333152 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.333099    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl" event={"ID":"251fbdae-e030-4ded-bda7-bb7e2dd09689","Type":"ContainerStarted","Data":"4c4b97b0efefe54cf803d053a928c197863447efaa0af63abcff486609258083"}
Jul 29 20:48:01.333261 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.333200    2569 patch_prober.go:28] interesting pod/downloads-7c97dc4555-5j86x container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" start-of-body=
Jul 29 20:48:01.333343 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.333261    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-5j86x" podUID="cf091058-cc54-4fdf-92fc-601a36e78ea9" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused"
Jul 29 20:48:01.334759 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.334701    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"6aeaec81d9b137708990c67312ef7f3c8308af19b29e596bde8c49ad21359a19"}
Jul 29 20:48:01.348927 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.347848    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nxs7w" podStartSLOduration=10.197059208 podStartE2EDuration="11.347829322s" totalImagesPullingTime="1.077269073s" totalInitContainerRuntime="73.501041ms" isStatefulPod=true podCreationTimestamp="2026-07-29 20:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:01.345299169 +0000 UTC m=+93.943275511" watchObservedRunningTime="2026-07-29 20:48:01.347829322 +0000 UTC m=+93.945805680"
Jul 29 20:48:01.365715 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:01.365646    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-5j86x" podStartSLOduration=3.49412918 podStartE2EDuration="22.365627462s" totalImagesPullingTime="18.871498282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:01.364147806 +0000 UTC m=+93.962124163" watchObservedRunningTime="2026-07-29 20:48:01.365627462 +0000 UTC m=+93.963603824"
Jul 29 20:48:02.108148 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:02.108110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:02.211806 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:02.211698    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:02.350561 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:02.350461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:02.356306 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:02.356046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-5j86x"
Jul 29 20:48:02.454689 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:02.454488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:05.352989 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.352908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl" event={"ID":"251fbdae-e030-4ded-bda7-bb7e2dd09689","Type":"ContainerStarted","Data":"8ee34213f6dbde7d49ce2e09f390a8f9c88f70c8e099a2a6038d14fc79c44220"}
Jul 29 20:48:05.353459 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.353102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:48:05.355634 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.355571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"624ce9d26d5765fd99750dd2669760768335ca51760c0ec559c0b7c83d40a79c"}
Jul 29 20:48:05.355893 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.355867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"177cbc7cb9b0622408c9ecdb4d7d8730d1d975f6fb517542f7741ed7e82ab698"}
Jul 29 20:48:05.356011 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.355923    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"a72c0813f0d9979d32f96de4222999488cd98d8e12cb3f4bd7218c3501ca5eef"}
Jul 29 20:48:05.357907 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.357881    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b" exitCode=0
Jul 29 20:48:05.358005 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.357943    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"}
Jul 29 20:48:05.359142 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.359122    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl"
Jul 29 20:48:05.363129 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.363094    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:05.369718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.369669    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-kfcjl" podStartSLOduration=6.733801682 podStartE2EDuration="10.369655174s" totalImagesPullingTime="3.635853492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:05.368368785 +0000 UTC m=+97.966345144" watchObservedRunningTime="2026-07-29 20:48:05.369655174 +0000 UTC m=+97.967631531"
Jul 29 20:48:05.464461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:05.464424    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:06.370483 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:06.370387    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"ea0354e1e16942df32d7132348fc3f64659a72c0ab9e23e96f931e773438f833"}
Jul 29 20:48:07.379335 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:07.379298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"c57f73cc9dd5f856ace7ada4d3578fe5116241b0b4090a38b7f616197241ee73"}
Jul 29 20:48:07.379889 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:07.379371    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" event={"ID":"84266f12-a4b3-4d83-8528-4f101873b7ec","Type":"ContainerStarted","Data":"4c79411f8b1c9c78d54684ff03e922b9fa74cef86ac23a00498db7ac7e972ea2"}
Jul 29 20:48:07.401687 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:07.401632    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs" podStartSLOduration=10.370992609 podStartE2EDuration="15.401612709s" totalImagesPullingTime="5.0306201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:07.400005907 +0000 UTC m=+99.997982284" watchObservedRunningTime="2026-07-29 20:48:07.401612709 +0000 UTC m=+99.999589066"
Jul 29 20:48:07.937494 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:07.937455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:08.035915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:08.035870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:08.385256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:08.385217    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:48:08.722160 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:08.722077    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:09.395874 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:09.395850    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-97c7954bb-vkjvs"
Jul 29 20:48:09.734553 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:09.734526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:09.804095 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:09.804068    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:10.397202 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397159    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"}
Jul 29 20:48:10.397556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397249    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"}
Jul 29 20:48:10.397556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397290    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"}
Jul 29 20:48:10.397556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"}
Jul 29 20:48:10.397556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"}
Jul 29 20:48:10.397556 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.397416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerStarted","Data":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"}
Jul 29 20:48:10.425142 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:10.425099    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.740983508 podStartE2EDuration="13.425083027s" totalImagesPullingTime="7.656905673s" totalInitContainerRuntime="27.193846ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:47:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:48:10.423529824 +0000 UTC m=+103.021506191" watchObservedRunningTime="2026-07-29 20:48:10.425083027 +0000 UTC m=+103.023059383"
Jul 29 20:48:11.424816 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:11.424778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:11.474248 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:11.474222    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:12.421450 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:12.421418    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:48:13.240021 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:13.239994    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lpmln"
Jul 29 20:48:13.295032 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:13.295006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:13.405981 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:13.405951    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:15.815594 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:15.815556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:15.991281 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:15.991238    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:23.206878 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:23.206838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:24.863131 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:24.863106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sfgfh_03c2ee14-ab98-46df-96f7-312d4fd0b652/serve-healthcheck-canary/0.log"
Jul 29 20:48:30.811065 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:30.811032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:30.865596 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:30.865567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:38.168470 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:38.168437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:38.269827 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:38.269725    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:38.325506 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:38.325471    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:48:57.420647 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:57.420610    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:48:57.440523 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:57.440493    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:48:57.596240 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:48:57.596212    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:06.915273 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:06.915237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:06.963670 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:06.963641    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:09.299135 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:09.299095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:09.359209 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:09.359177    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:15.393615 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.393574    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:15.394112 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394041    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60" gracePeriod=600
Jul 29 20:49:15.394249 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394211    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-web" containerID="cri-o://8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f" gracePeriod=600
Jul 29 20:49:15.394333 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394258    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy" containerID="cri-o://8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc" gracePeriod=600
Jul 29 20:49:15.394333 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394281    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="config-reloader" containerID="cri-o://e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1" gracePeriod=600
Jul 29 20:49:15.394433 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394216    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="prometheus" containerID="cri-o://d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c" gracePeriod=600
Jul 29 20:49:15.394433 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.394376    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="thanos-sidecar" containerID="cri-o://e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f" gracePeriod=600
Jul 29 20:49:15.640077 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.639775    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651121    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651144    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651155    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651162    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651169    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651177    2569 generic.go:356] "Generic (PLEG): container finished" podID="3ff26dae-8b90-4475-8641-ef5404631c53" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c" exitCode=0
Jul 29 20:49:15.651193 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651186    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651197    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651349    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651367    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651396    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"}
Jul 29 20:49:15.651482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.651421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3ff26dae-8b90-4475-8641-ef5404631c53","Type":"ContainerDied","Data":"0387e273941fc09713a648db57c4fbf7abfc4c6436f2f85c957c2cb9f5a7e87d"}
Jul 29 20:49:15.662711 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.662686    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.664841 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.664915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664857    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.664915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664877    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.664915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664896    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664933    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664952    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664967    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.664985    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665004    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665021    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665036    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgmw7\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.665061 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665054    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665095    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665120    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665170    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665227    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665263    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665264    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665288    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0\") pod \"3ff26dae-8b90-4475-8641-ef5404631c53\" (UID: \"3ff26dae-8b90-4475-8641-ef5404631c53\") "
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.665563    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.666965    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.669403    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.669806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.670121    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.670175    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.673461 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.670414    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:49:15.675729 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.671052    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:49:15.675729 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.672907    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:49:15.675729 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.673246    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out" (OuterVolumeSpecName: "config-out") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:49:15.675729 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.675714    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config" (OuterVolumeSpecName: "config") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.676036 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.675884    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.676927 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.676899    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.677056 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.676946    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.677056 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.676968    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.677056 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.677004    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.678986 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.678863    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.678986 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.678893    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.678986 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.678926    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7" (OuterVolumeSpecName: "kube-api-access-vgmw7") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "kube-api-access-vgmw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:49:15.679649 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.679386    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.689699 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.689437    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.691383 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.691356    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config" (OuterVolumeSpecName: "web-config") pod "3ff26dae-8b90-4475-8641-ef5404631c53" (UID: "3ff26dae-8b90-4475-8641-ef5404631c53"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:49:15.697590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.697575    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.704547 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.704526    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.711837 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.711810    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.718521 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.718496    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.718791 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.718770    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.718866 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.718799    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.718866 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.718833    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.719081 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.719062    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.719145 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719091    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.719145 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719107    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.719304 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.719287    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.719362 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719306    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.719362 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719319    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.719506 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.719491    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.719560 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719509    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.719560 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719520    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.719778 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.719756    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.719860 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719783    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.719860 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.719804    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.720107 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.720076    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.720178 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720116    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.720178 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720136    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.720400 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:49:15.720380    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.720505 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720412    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.720505 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720430    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.720717 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720698    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.720802 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720719    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.720943 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720919    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.721043 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.720943    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.721217 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721196    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.721281 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721220    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.721500 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721473    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.721596 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721500    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.721787 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721768    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.721872 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.721788    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.722044 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722024    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.722124 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722045    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.722303 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722277    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.722396 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722303    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.722558 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722540    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.722604 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722559    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.722789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722768    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.722789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722789    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.723009 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.722992    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.723060 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723009    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.723236 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723218    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.723276 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723237    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.723462 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723437    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.723506 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723461    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.723696 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723677    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.723763 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723698    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.723899 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723879    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.723953 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.723899    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.724150 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724119    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.724150 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724142    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.724379 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724360    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.724432 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724380    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.724603 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724585    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.724647 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724605    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.724849 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724831    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.724913 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.724849    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.725055 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725037    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.725109 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725056    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.725290 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725273    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.725330 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725290    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.725480 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725465    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.725525 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725480    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.725702 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725680    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.725785 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725705    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.725951 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725935    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.726008 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.725951    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.726164 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726148    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.726216 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726164    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.726387 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726372    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.726435 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726388    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.726598 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726582    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.726598 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726598    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.726822 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726806    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.726822 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.726821    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.727030 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727011    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.727030 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727030    2569 scope.go:117] "RemoveContainer" containerID="fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"
Jul 29 20:49:15.727241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727220    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60"} err="failed to get container status \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": rpc error: code = NotFound desc = could not find container \"fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60\": container with ID starting with fd681a45fa72febc0b121837230e6606d58ab2be48ce6ac24608c6c0dc1f8e60 not found: ID does not exist"
Jul 29 20:49:15.727284 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727242    2569 scope.go:117] "RemoveContainer" containerID="8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"
Jul 29 20:49:15.727486 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727468    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc"} err="failed to get container status \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": rpc error: code = NotFound desc = could not find container \"8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc\": container with ID starting with 8154313f360f543266d4811dffede2794b40ac797d319a7916e077f766bdf2fc not found: ID does not exist"
Jul 29 20:49:15.727527 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727487    2569 scope.go:117] "RemoveContainer" containerID="8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"
Jul 29 20:49:15.727722 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727698    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f"} err="failed to get container status \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": rpc error: code = NotFound desc = could not find container \"8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f\": container with ID starting with 8c1350ecc9ea8a3b59a02a0eb7cd81e82a461ddc51e979533602cc876f6abc3f not found: ID does not exist"
Jul 29 20:49:15.727722 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727721    2569 scope.go:117] "RemoveContainer" containerID="e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"
Jul 29 20:49:15.727963 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727946    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f"} err="failed to get container status \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": rpc error: code = NotFound desc = could not find container \"e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f\": container with ID starting with e69761de0072946e5bb4b4044a5e26d5d5f2b581797bf43c319927f3f7af847f not found: ID does not exist"
Jul 29 20:49:15.727963 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.727962    2569 scope.go:117] "RemoveContainer" containerID="e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"
Jul 29 20:49:15.728171 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.728156    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1"} err="failed to get container status \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": rpc error: code = NotFound desc = could not find container \"e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1\": container with ID starting with e7996c41acc212876e9e90bcbb33b3660b4d6dede78f7d008d0e97c698c7bbf1 not found: ID does not exist"
Jul 29 20:49:15.728171 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.728170    2569 scope.go:117] "RemoveContainer" containerID="d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"
Jul 29 20:49:15.728393 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.728373    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c"} err="failed to get container status \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": rpc error: code = NotFound desc = could not find container \"d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c\": container with ID starting with d67430d9275c8ec3eede14b2d6b8c8c29ef6436c54faa5abfbec7871a3ef6c7c not found: ID does not exist"
Jul 29 20:49:15.728393 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.728393    2569 scope.go:117] "RemoveContainer" containerID="80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"
Jul 29 20:49:15.728596 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.728581    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b"} err="failed to get container status \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": rpc error: code = NotFound desc = could not find container \"80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b\": container with ID starting with 80aacc4dc71aec6a24fcac24395ae747d1d6c65401e111fdf0df27ea4d8a911b not found: ID does not exist"
Jul 29 20:49:15.766139 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766116    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-kube-rbac-proxy\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766139 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766135    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-metrics-client-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766144    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgmw7\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-kube-api-access-vgmw7\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766154    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766164    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766173    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766181    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766190    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766198    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-web-config\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766205    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-db\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766214    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-config\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766222    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3ff26dae-8b90-4475-8641-ef5404631c53-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766230    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3ff26dae-8b90-4475-8641-ef5404631c53-config-out\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766237    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766246    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-grpc-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766256 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766254    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766642 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766263    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766642 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766272    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3ff26dae-8b90-4475-8641-ef5404631c53-tls-assets\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.766642 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.766280    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3ff26dae-8b90-4475-8641-ef5404631c53-secret-metrics-client-certs\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 20:49:15.967524 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.967475    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:15.974614 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.974592    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:15.997994 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.997969    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:15.998245 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998230    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-web"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998246    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-web"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998257    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="prometheus"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998263    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="prometheus"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998270    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="thanos-sidecar"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998276    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="thanos-sidecar"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998282    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="config-reloader"
Jul 29 20:49:15.998285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998287    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="config-reloader"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998293    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998300    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998317    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="init-config-reloader"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998323    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="init-config-reloader"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998345    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-thanos"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998351    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-thanos"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998406    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998414    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-thanos"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998420    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="config-reloader"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998427    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="prometheus"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998432    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="kube-rbac-proxy-web"
Jul 29 20:49:15.998482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:15.998437    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" containerName="thanos-sidecar"
Jul 29 20:49:16.001913 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.001897    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.004126 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004099    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 20:49:16.004416 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004402    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 20:49:16.004577 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004558    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 20:49:16.004656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004645    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 20:49:16.004798 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004724    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 20:49:16.004798 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.004769    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005009    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005035    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005045    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005059    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-fb3ih3dqf31v9\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005040    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 20:49:16.005161 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005016    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v6dgd\""
Jul 29 20:49:16.005520 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 20:49:16.005584 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005505    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 20:49:16.005644 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.005592    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 20:49:16.008478 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.008411    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 20:49:16.012273 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.012254    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 20:49:16.016726 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.016706    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:16.067633 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067645    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067718 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067817 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067817 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067817 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067817 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.067930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068042 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067939    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068042 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.067961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068042 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068040    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068085    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bk6v\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-kube-api-access-9bk6v\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068123    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068244 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.068244 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.068152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168421 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168421 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168397    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168421 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bk6v\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-kube-api-access-9bk6v\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168488    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168553    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.168623 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168686    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168726    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168817    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168862    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169020 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.168891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169469 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.169246    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169469 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.169272    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.169568 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.169546    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.171456 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.169780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.171650 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.171627    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-config-out\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.171825 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.171800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.171958 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.171934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172029 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.171984    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172029 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.171984    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172141 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.172124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172438 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.172413    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172569 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.172509    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172677 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.172583    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-web-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.172880 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.172852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.173576 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.173549    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/503bd6c7-1043-4c90-a258-6d131606cdfa-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.174037 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.174013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.174415 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.174389    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.174415 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.174396    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-config\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.174656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.174641    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/503bd6c7-1043-4c90-a258-6d131606cdfa-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.177130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.177109    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bk6v\" (UniqueName: \"kubernetes.io/projected/503bd6c7-1043-4c90-a258-6d131606cdfa-kube-api-access-9bk6v\") pod \"prometheus-k8s-0\" (UID: \"503bd6c7-1043-4c90-a258-6d131606cdfa\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.313553 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.313525    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:16.441533 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.441499    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 20:49:16.445489 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:49:16.445458    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503bd6c7_1043_4c90_a258_6d131606cdfa.slice/crio-0f13dda2272e6cd868129fb0ba7419764c788c14b2aca07530e240940fb9b0a6 WatchSource:0}: Error finding container 0f13dda2272e6cd868129fb0ba7419764c788c14b2aca07530e240940fb9b0a6: Status 404 returned error can't find the container with id 0f13dda2272e6cd868129fb0ba7419764c788c14b2aca07530e240940fb9b0a6
Jul 29 20:49:16.663289 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.663259    2569 generic.go:356] "Generic (PLEG): container finished" podID="503bd6c7-1043-4c90-a258-6d131606cdfa" containerID="eb48e472f84b2b5f0695d17dfc87e1dc0bc75c0365f133ed3a8f1793a0709876" exitCode=0
Jul 29 20:49:16.663404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.663336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerDied","Data":"eb48e472f84b2b5f0695d17dfc87e1dc0bc75c0365f133ed3a8f1793a0709876"}
Jul 29 20:49:16.663446 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:16.663439    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"0f13dda2272e6cd868129fb0ba7419764c788c14b2aca07530e240940fb9b0a6"}
Jul 29 20:49:17.671864 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"9f601d6900c28d49da9b64c3829d9c8eb2dfca21f5fc1028a31f370941d14801"}
Jul 29 20:49:17.671864 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"3f00bf3413f1471e05512e6dc882efe52abadbfe6193f1a1f1f21bcdcffa00e8"}
Jul 29 20:49:17.672251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"57d1284b43fd25d1c6e3263e5fc7451d3580bcfac84eabf55cca562bd3b5c610"}
Jul 29 20:49:17.672251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671918    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"45d2a2e3eb7fd4c1c34b028cfada7b9024169be0af50509e3c5dd0e2bd1feefc"}
Jul 29 20:49:17.672251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671942    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"ce886d0d7f7cc14c718fd3ee33057a24563843e857c2f53c867351f2a4a72636"}
Jul 29 20:49:17.672251 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.671965    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"503bd6c7-1043-4c90-a258-6d131606cdfa","Type":"ContainerStarted","Data":"e989a5cc20a746ea9a8dd2ea35b6c0d47b50af3f2bac59585773daa8d22ff203"}
Jul 29 20:49:17.699201 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.699140    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.679767576 podStartE2EDuration="2.699119807s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.352231ms" isStatefulPod=false podCreationTimestamp="2026-07-29 20:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:49:17.69798723 +0000 UTC m=+170.295963578" watchObservedRunningTime="2026-07-29 20:49:17.699119807 +0000 UTC m=+170.297096165"
Jul 29 20:49:17.896656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:17.896624    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ff26dae-8b90-4475-8641-ef5404631c53" path="/var/lib/kubelet/pods/3ff26dae-8b90-4475-8641-ef5404631c53/volumes"
Jul 29 20:49:18.838427 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:18.838391    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:18.891359 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:18.891328    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:20.615631 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:20.615551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:20.716200 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:20.716166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:20.774810 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:20.774786    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:21.313823 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:21.313779    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:49:30.813642 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:30.813600    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:30.914632 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:30.914604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:55.115881 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:55.115851    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:55.171043 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:55.171018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:49:57.789673 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:49:57.789636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:08.662135 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:08.662101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:08.763131 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:08.763100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:16.314066 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:16.314017    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:50:16.330258 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:16.330226    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:50:16.916989 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:16.916961    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 20:50:35.366071 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:35.366038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:36.146830 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:36.146791    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:38.777412 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:38.777382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:38.875343 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:38.875308    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:40.247643 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:40.247610    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:40.370465 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:40.370430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:42.676188 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:42.676157    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:42.860322 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:42.860291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:45.526856 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:45.526803    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:45.583453 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:45.583419    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:50.455810 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:50.455729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:51.561439 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:51.561398    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:51.652993 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:51.652961    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:57.548821 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:57.548789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:57.649307 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:57.649274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:50:59.519748 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:50:59.519700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:13.290723 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:13.290688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:14.563017 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.562976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:14.671234 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.671203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:14.729007 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.728973    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"]
Jul 29 20:51:14.731503 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.731483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.735305 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.735281    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 20:51:14.735413 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.735332    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 20:51:14.735695 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.735678    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 20:51:14.735798 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.735778    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x7p5x\""
Jul 29 20:51:14.735844 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.735798    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 20:51:14.747595 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.747576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"]
Jul 29 20:51:14.896133 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.896044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-manifests\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.896133 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.896092    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.896133 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.896120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.896353 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.896142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p4xh\" (UniqueName: \"kubernetes.io/projected/096a549e-855d-439d-a839-755ad9b7387d-kube-api-access-8p4xh\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.896353 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.896237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-tmp\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.907304 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.907282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:14.996637 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.996601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-tmp\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.996834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.996646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-manifests\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.996834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.996677    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.996834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.996699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.996834 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.996716    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p4xh\" (UniqueName: \"kubernetes.io/projected/096a549e-855d-439d-a839-755ad9b7387d-kube-api-access-8p4xh\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.997160 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.997131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-manifests\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.999022 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.999000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/096a549e-855d-439d-a839-755ad9b7387d-tmp\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.999244 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.999228    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:14.999306 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:14.999267    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/096a549e-855d-439d-a839-755ad9b7387d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:15.013241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.013215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:15.013241 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.013234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p4xh\" (UniqueName: \"kubernetes.io/projected/096a549e-855d-439d-a839-755ad9b7387d-kube-api-access-8p4xh\") pod \"opendatahub-operator-controller-manager-7dbc94cc7-h954v\" (UID: \"096a549e-855d-439d-a839-755ad9b7387d\") " pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:15.041089 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.041065    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:15.123224 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.123193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:15.172156 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.172131    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"]
Jul 29 20:51:15.175305 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:51:15.175277    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096a549e_855d_439d_a839_755ad9b7387d.slice/crio-8a2436529c0eccf9037e2523dbb4577bb6a41556f0abdbe9bdedb11357baddc4 WatchSource:0}: Error finding container 8a2436529c0eccf9037e2523dbb4577bb6a41556f0abdbe9bdedb11357baddc4: Status 404 returned error can't find the container with id 8a2436529c0eccf9037e2523dbb4577bb6a41556f0abdbe9bdedb11357baddc4
Jul 29 20:51:15.209086 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.209050    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:15.329832 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.329798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:15.436930 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:15.436850    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:16.124683 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:16.124637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v" event={"ID":"096a549e-855d-439d-a839-755ad9b7387d","Type":"ContainerStarted","Data":"8a2436529c0eccf9037e2523dbb4577bb6a41556f0abdbe9bdedb11357baddc4"}
Jul 29 20:51:17.778890 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:17.778848    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:17.885259 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:17.885193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:19.139184 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:19.139146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v" event={"ID":"096a549e-855d-439d-a839-755ad9b7387d","Type":"ContainerStarted","Data":"0505fcdd965137f14f096d664a5798df1ece7ed7b36c477bec5434487edc85d5"}
Jul 29 20:51:20.145473 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:20.145436    2569 generic.go:356] "Generic (PLEG): container finished" podID="096a549e-855d-439d-a839-755ad9b7387d" containerID="0505fcdd965137f14f096d664a5798df1ece7ed7b36c477bec5434487edc85d5" exitCode=0
Jul 29 20:51:20.145871 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:20.145522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v" event={"ID":"096a549e-855d-439d-a839-755ad9b7387d","Type":"ContainerDied","Data":"0505fcdd965137f14f096d664a5798df1ece7ed7b36c477bec5434487edc85d5"}
Jul 29 20:51:21.151484 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:21.151452    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v" event={"ID":"096a549e-855d-439d-a839-755ad9b7387d","Type":"ContainerStarted","Data":"a4c2cc5fac5ada6f6280073d61dc3b8538c424bede87df09b6402c05dcbb8131"}
Jul 29 20:51:21.151872 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:21.151659    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:21.178281 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:21.178230    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v" podStartSLOduration=2.866263334 podStartE2EDuration="7.178215111s" totalImagesPullingTime="3.28968047s" totalInitContainerRuntime="1.022271307s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:21.177070218 +0000 UTC m=+293.775046610" watchObservedRunningTime="2026-07-29 20:51:21.178215111 +0000 UTC m=+293.776191525"
Jul 29 20:51:27.756743 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:27.756701    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 20:51:32.158501 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.158471    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7dbc94cc7-h954v"
Jul 29 20:51:32.204796 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.204761    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"]
Jul 29 20:51:32.206983 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.206961    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.209901 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.209877    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 20:51:32.210028 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.209881    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mhtvw\""
Jul 29 20:51:32.210028 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.210017    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 20:51:32.210147 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.210065    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 20:51:32.210147 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.209963    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 20:51:32.210147 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.209965    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 20:51:32.217473 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.217449    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"]
Jul 29 20:51:32.227535 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.227507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgmtd\" (UniqueName: \"kubernetes.io/projected/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-kube-api-access-cgmtd\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.227636 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.227551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.227710 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.227688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-metrics-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.227819 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.227802    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-manager-config\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.328358 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.328325    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-manager-config\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.328546 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.328377    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgmtd\" (UniqueName: \"kubernetes.io/projected/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-kube-api-access-cgmtd\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.328546 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.328405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.328546 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.328437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-metrics-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.329111 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.329082    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-manager-config\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.330876 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.330852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.330968 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.330887    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-metrics-cert\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.336832 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.336798    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgmtd\" (UniqueName: \"kubernetes.io/projected/6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb-kube-api-access-cgmtd\") pod \"lws-controller-manager-6896cc89b8-rbvx9\" (UID: \"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb\") " pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.518265 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.518236    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:32.640048 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.640027    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"]
Jul 29 20:51:32.642689 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:51:32.642659    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6575cb1c_f4b1_46a1_a5c6_d9f1abd8b6eb.slice/crio-5b0aea8167b9218e0b1498ffef20a1fff52bae14de7bf18345e7bbbd31614452 WatchSource:0}: Error finding container 5b0aea8167b9218e0b1498ffef20a1fff52bae14de7bf18345e7bbbd31614452: Status 404 returned error can't find the container with id 5b0aea8167b9218e0b1498ffef20a1fff52bae14de7bf18345e7bbbd31614452
Jul 29 20:51:32.644616 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:32.644599    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:51:33.201379 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:33.201343    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9" event={"ID":"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb","Type":"ContainerStarted","Data":"5b0aea8167b9218e0b1498ffef20a1fff52bae14de7bf18345e7bbbd31614452"}
Jul 29 20:51:33.814949 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:33.814914    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:33.990872 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:33.990833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:34.521162 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.521121    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:34.532750 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.532704    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"]
Jul 29 20:51:34.539226 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.539204    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.543338 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.543299    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 20:51:34.543847 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.543811    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 20:51:34.543847 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.543822    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 20:51:34.544015 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.543888    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 20:51:34.544101 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.544086    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-728nc\""
Jul 29 20:51:34.556393 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.556373    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"]
Jul 29 20:51:34.650296 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.650255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tmp\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.650471 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.650321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdvvw\" (UniqueName: \"kubernetes.io/projected/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-kube-api-access-fdvvw\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.650471 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.650372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tls-certs\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.698863 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.698823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:34.752308 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.752177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tmp\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.752308 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.752248    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdvvw\" (UniqueName: \"kubernetes.io/projected/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-kube-api-access-fdvvw\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.752308 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.752303    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tls-certs\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.755003 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.754971    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tmp\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.755169 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.755147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-tls-certs\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.769707 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.769667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdvvw\" (UniqueName: \"kubernetes.io/projected/ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08-kube-api-access-fdvvw\") pod \"kube-auth-proxy-5b585ddb49-whcx8\" (UID: \"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08\") " pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.796630 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.796549    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:34.851847 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.851814    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"
Jul 29 20:51:34.863774 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.863227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:34.970662 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:34.970626    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:35.218796 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.218751    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:35.321693 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.321672    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8"]
Jul 29 20:51:35.324427 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:51:35.324399    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe03c35_f80e_4e05_b1cb_a3f0b1bafd08.slice/crio-513479a3aa9841f6d65bf9dda2b2157805a8d0aacaef8dfd71dfc2abf521bf58 WatchSource:0}: Error finding container 513479a3aa9841f6d65bf9dda2b2157805a8d0aacaef8dfd71dfc2abf521bf58: Status 404 returned error can't find the container with id 513479a3aa9841f6d65bf9dda2b2157805a8d0aacaef8dfd71dfc2abf521bf58
Jul 29 20:51:35.602342 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.602309    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:35.820113 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.820079    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:35.882404 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.882330    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:35.934547 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:35.934512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:36.214690 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:36.214610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9" event={"ID":"6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb","Type":"ContainerStarted","Data":"f40623eefd1c34f9ea147cebd8a25511184f7dabc52249681fada3cc85853922"}
Jul 29 20:51:36.214878 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:36.214690    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:36.215761 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:36.215717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8" event={"ID":"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08","Type":"ContainerStarted","Data":"513479a3aa9841f6d65bf9dda2b2157805a8d0aacaef8dfd71dfc2abf521bf58"}
Jul 29 20:51:36.229564 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:36.229519    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9" podStartSLOduration=1.64874441 podStartE2EDuration="4.229505732s" totalImagesPullingTime="2.580761322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:36.22920728 +0000 UTC m=+308.827183637" watchObservedRunningTime="2026-07-29 20:51:36.229505732 +0000 UTC m=+308.827482090"
Jul 29 20:51:36.753368 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:36.753331    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:37.894976 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:37.894946    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:37.949841 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:37.949778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:38.226920 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:38.226885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8" event={"ID":"ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08","Type":"ContainerStarted","Data":"e56f681cde66162298ce29a839a72fd3967749b7e399697f9d1066bef4239687"}
Jul 29 20:51:38.245490 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:38.245443    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b585ddb49-whcx8" podStartSLOduration=1.895430293 podStartE2EDuration="4.24542833s" totalImagesPullingTime="2.349998037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:51:38.244635608 +0000 UTC m=+310.842611988" watchObservedRunningTime="2026-07-29 20:51:38.24542833 +0000 UTC m=+310.843404687"
Jul 29 20:51:41.357219 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:41.357188    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:41.633295 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:41.633218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:41.697282 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:41.697252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:41.987244 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:41.987209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:42.144880 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:42.144850    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:42.472329 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:42.472246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:45.844868 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:45.844836    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:45.934252 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:45.934192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.182460 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:46.182371    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:46.269766 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:46.269721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:47.210993 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:47.210955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:47.223601 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:47.223575    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6896cc89b8-rbvx9"
Jul 29 20:51:47.294769 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:47.294721    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:47.352422 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:47.352389    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.366257 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:56.366223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:56.466680 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:56.466646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:58.223238 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:58.223201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:51:58.282177 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:51:58.282144    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:35.247649 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:52:35.247567    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:35.344167 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:52:35.344131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:35.411033 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:52:35.411004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:36.154230 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:52:36.154193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:52:36.375015 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:52:36.374983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:03.186281 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:03.186240    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:03.286502 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:03.286470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:04.866846 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:04.866809    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:08.453290 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:08.453262    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:11.478049 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:11.478012    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:12.919463 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:12.919430    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:13.258096 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:13.258068    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:13.856194 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:13.856158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:13.967414 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:13.967381    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:14.084285 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:14.084259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:16.309042 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:16.309009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:16.386698 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:16.386663    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:17.289571 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:17.289538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:19.100232 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:19.100200    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:20.654334 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:20.654298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:20.754680 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:20.754644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:22.087049 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.087013    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"]
Jul 29 20:53:22.089259 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.089243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.091643 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.091622    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 20:53:22.091789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.091679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 20:53:22.091789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.091622    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 20:53:22.091789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.091760    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wdbm6\""
Jul 29 20:53:22.092528 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.092508    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 20:53:22.098343 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.098320    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"]
Jul 29 20:53:22.100787 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.100767    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:22.118364 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.118341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.118444 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.118368    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ff415ce-f7eb-4751-997a-b39a7ac75fad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.118444 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.118406    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qthg2\" (UniqueName: \"kubernetes.io/projected/5ff415ce-f7eb-4751-997a-b39a7ac75fad-kube-api-access-qthg2\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.218946 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.218920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qthg2\" (UniqueName: \"kubernetes.io/projected/5ff415ce-f7eb-4751-997a-b39a7ac75fad-kube-api-access-qthg2\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.219063 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.218972    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.219063 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.218991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ff415ce-f7eb-4751-997a-b39a7ac75fad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.219181 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:53:22.219077    2569 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 29 20:53:22.219181 ip-10-0-139-241 kubenswrapper[2569]: E0729 20:53:22.219137    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert podName:5ff415ce-f7eb-4751-997a-b39a7ac75fad nodeName:}" failed. No retries permitted until 2026-07-29 20:53:22.719119092 +0000 UTC m=+415.317095430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-qjskm" (UID: "5ff415ce-f7eb-4751-997a-b39a7ac75fad") : secret "plugin-serving-cert" not found
Jul 29 20:53:22.219508 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.219492    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ff415ce-f7eb-4751-997a-b39a7ac75fad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.234836 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.234809    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qthg2\" (UniqueName: \"kubernetes.io/projected/5ff415ce-f7eb-4751-997a-b39a7ac75fad-kube-api-access-qthg2\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.474001 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.473971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:22.722455 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.722425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:22.724866 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.724823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ff415ce-f7eb-4751-997a-b39a7ac75fad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-qjskm\" (UID: \"5ff415ce-f7eb-4751-997a-b39a7ac75fad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:23.000039 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:22.999974    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"
Jul 29 20:53:23.127976 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:23.127952    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm"]
Jul 29 20:53:23.130448 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:53:23.130417    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ff415ce_f7eb_4751_997a_b39a7ac75fad.slice/crio-7ecb90a4f8520b0a48c5b0ca174b49e3951d6067ab804cc3857f8dea1897a0e7 WatchSource:0}: Error finding container 7ecb90a4f8520b0a48c5b0ca174b49e3951d6067ab804cc3857f8dea1897a0e7: Status 404 returned error can't find the container with id 7ecb90a4f8520b0a48c5b0ca174b49e3951d6067ab804cc3857f8dea1897a0e7
Jul 29 20:53:23.664486 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:23.664453    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm" event={"ID":"5ff415ce-f7eb-4751-997a-b39a7ac75fad","Type":"ContainerStarted","Data":"7ecb90a4f8520b0a48c5b0ca174b49e3951d6067ab804cc3857f8dea1897a0e7"}
Jul 29 20:53:24.259708 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:24.259670    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:24.322217 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:24.322163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:33.487008 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:33.486974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:33.585590 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:33.585557    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:36.479519 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:36.479487    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:36.579849 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:36.579808    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:39.815704 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:39.815665    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:39.915914 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:39.915883    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:53:44.766019 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:44.765988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm" event={"ID":"5ff415ce-f7eb-4751-997a-b39a7ac75fad","Type":"ContainerStarted","Data":"4e844b242a986cfda769f1038c14d2c07eddcb9464fd37f5111ac0d1da973788"}
Jul 29 20:53:44.782864 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:53:44.782819    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-qjskm" podStartSLOduration=1.358625371 podStartE2EDuration="22.782805616s" totalImagesPullingTime="21.424180245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:53:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:53:44.781980987 +0000 UTC m=+437.379957345" watchObservedRunningTime="2026-07-29 20:53:44.782805616 +0000 UTC m=+437.380781973"
Jul 29 20:54:01.265347 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:01.265266    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:01.354925 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:01.354896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:07.060909 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.060876    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:07.182114 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.182084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:07.282583 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.282553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:07.560536 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.560502    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:54:07.602401 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.602370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:54:07.602401 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.602405    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:54:07.602591 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.602480    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.604803 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.604730    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 20:54:07.794940 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.794898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-config-file\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.795106 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.795039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mscwb\" (UniqueName: \"kubernetes.io/projected/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-kube-api-access-mscwb\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.891655 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.891561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:07.895465 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.895440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mscwb\" (UniqueName: \"kubernetes.io/projected/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-kube-api-access-mscwb\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.895605 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.895477    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-config-file\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.896127 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.896107    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-config-file\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.903086 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.903061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mscwb\" (UniqueName: \"kubernetes.io/projected/8013d924-e01f-4ca5-9bde-2e5ed1fdc288-kube-api-access-mscwb\") pod \"limitador-limitador-54d6d988d5-7rcwp\" (UID: \"8013d924-e01f-4ca5-9bde-2e5ed1fdc288\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.912627 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.912609    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:07.951105 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:07.951080    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:08.042515 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:54:08.042486    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8013d924_e01f_4ca5_9bde_2e5ed1fdc288.slice/crio-b244b0e438f1e144fe2c5ecaf8ab6e3df481d8b16254c40161de6637645836e2 WatchSource:0}: Error finding container b244b0e438f1e144fe2c5ecaf8ab6e3df481d8b16254c40161de6637645836e2: Status 404 returned error can't find the container with id b244b0e438f1e144fe2c5ecaf8ab6e3df481d8b16254c40161de6637645836e2
Jul 29 20:54:08.043698 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:08.043671    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:54:08.868121 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:08.868090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp" event={"ID":"8013d924-e01f-4ca5-9bde-2e5ed1fdc288","Type":"ContainerStarted","Data":"b244b0e438f1e144fe2c5ecaf8ab6e3df481d8b16254c40161de6637645836e2"}
Jul 29 20:54:12.896903 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:12.896870    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp" event={"ID":"8013d924-e01f-4ca5-9bde-2e5ed1fdc288","Type":"ContainerStarted","Data":"5d7a569ebd998cb25e6dae75e60d177552979ec41a897d3d51c80068d7fffc75"}
Jul 29 20:54:12.897314 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:12.896925    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:12.913107 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:12.913063    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp" podStartSLOduration=1.519316986 podStartE2EDuration="5.913049131s" totalImagesPullingTime="4.393732145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:54:12.910828197 +0000 UTC m=+465.508804556" watchObservedRunningTime="2026-07-29 20:54:12.913049131 +0000 UTC m=+465.511025487"
Jul 29 20:54:23.901410 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:23.901380    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"
Jul 29 20:54:28.229924 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:28.229890    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:28.330321 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:28.330291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:28.724504 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:28.724439    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:40.645089 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:40.645051    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:40.919194 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:40.919104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:41.770477 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:41.770440    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:47.376409 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:47.376372    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:47.479597 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:47.479563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:47.610021 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:47.609990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:50.569727 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:50.569690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:54:50.667858 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:54:50.667827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:02.418576 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:02.418539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:02.494058 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:02.494032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:03.901697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:03.901664    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:03.953396 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:03.953374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:04.132501 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:04.132472    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:22.207454 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:22.207376    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:22.268045 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:22.268015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:23.897575 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:23.897538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:24.373130 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:24.373086    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:24.469562 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:24.469529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:24.564091 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:24.564063    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:24.835861 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:24.835829    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:55:35.631572 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:35.631538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:35.730712 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:35.730687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:46.670858 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:46.670828    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:47.254769 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:47.254719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:55:48.427869 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:48.427830    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:55:48.527970 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:55:48.527935    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:03.844121 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.844086    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"]
Jul 29 20:56:03.846666 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.846646    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.850079 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.850057    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 20:56:03.850079 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.850070    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 20:56:03.850232 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.850080    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t86zw\""
Jul 29 20:56:03.850232 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.850064    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 29 20:56:03.859199 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.859167    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"]
Jul 29 20:56:03.879192 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.879156    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:03.970397 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.970397 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.970656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ngzv\" (UniqueName: \"kubernetes.io/projected/815a0c7e-6733-45dc-86d6-de441fd4af96-kube-api-access-5ngzv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.970656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970515    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/815a0c7e-6733-45dc-86d6-de441fd4af96-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.970656 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970546    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:03.970821 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:03.970648    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.071649 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ngzv\" (UniqueName: \"kubernetes.io/projected/815a0c7e-6733-45dc-86d6-de441fd4af96-kube-api-access-5ngzv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.071883 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071664    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/815a0c7e-6733-45dc-86d6-de441fd4af96-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.071883 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.071883 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.071883 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.072082 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.071921    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.072200 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.072172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.072259 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.072226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.072310 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.072263    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.074032 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.074015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/815a0c7e-6733-45dc-86d6-de441fd4af96-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.074316 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.074298    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/815a0c7e-6733-45dc-86d6-de441fd4af96-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.079707 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.079682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ngzv\" (UniqueName: \"kubernetes.io/projected/815a0c7e-6733-45dc-86d6-de441fd4af96-kube-api-access-5ngzv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6xkck\" (UID: \"815a0c7e-6733-45dc-86d6-de441fd4af96\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.157955 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.157869    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:04.282630 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.282603    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"]
Jul 29 20:56:04.284701 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:56:04.284676    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815a0c7e_6733_45dc_86d6_de441fd4af96.slice/crio-bd69a41cc4aac4c373207cdd063fa0a98a3df39e691afd592a1be82da656a4e9 WatchSource:0}: Error finding container bd69a41cc4aac4c373207cdd063fa0a98a3df39e691afd592a1be82da656a4e9: Status 404 returned error can't find the container with id bd69a41cc4aac4c373207cdd063fa0a98a3df39e691afd592a1be82da656a4e9
Jul 29 20:56:04.408492 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.408433    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck" event={"ID":"815a0c7e-6733-45dc-86d6-de441fd4af96","Type":"ContainerStarted","Data":"bd69a41cc4aac4c373207cdd063fa0a98a3df39e691afd592a1be82da656a4e9"}
Jul 29 20:56:04.545099 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:04.545073    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:56:05.414073 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:05.414041    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck" event={"ID":"815a0c7e-6733-45dc-86d6-de441fd4af96","Type":"ContainerStarted","Data":"883f92365eeab30d93f324275f9ad5e578f8d0dc473180bedb6220f662da9d45"}
Jul 29 20:56:05.430507 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:05.430460    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck" podStartSLOduration=1.454108283 podStartE2EDuration="2.430445752s" totalImagesPullingTime="976.337469ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:56:05.429289782 +0000 UTC m=+578.027266140" watchObservedRunningTime="2026-07-29 20:56:05.430445752 +0000 UTC m=+578.028422109"
Jul 29 20:56:14.158474 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:14.158438    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:14.159082 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:14.158499    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:14.171097 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:14.171067    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:14.470409 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:14.470338    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6xkck"
Jul 29 20:56:33.480915 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:33.480878    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:34.650476 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:34.650442    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:56:37.569164 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:37.569121    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:39.445053 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:39.445017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:56:52.641119 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:52.641045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:55.035789 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.035755    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"]
Jul 29 20:56:55.038707 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.038686    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.041045 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.041015    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 20:56:55.047447 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.047427    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"]
Jul 29 20:56:55.086899 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.086876    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:56:55.104980 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.104954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjrt8\" (UniqueName: \"kubernetes.io/projected/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-kube-api-access-rjrt8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.105057 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.104996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.105106 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.105068    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.105106 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.105096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.105202 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.105116    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.105240 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.105206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.205774 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205752    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.205924 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205791    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.205924 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrt8\" (UniqueName: \"kubernetes.io/projected/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-kube-api-access-rjrt8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.205924 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.205924 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.206122 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.205995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.206191 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.206172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.206228 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.206192    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.206313 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.206295    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.207814 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.207795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.208106 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.208087    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.216022 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.216000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjrt8\" (UniqueName: \"kubernetes.io/projected/3e83f419-99fa-4d38-a9cf-01d3230a5cb8-kube-api-access-rjrt8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7\" (UID: \"3e83f419-99fa-4d38-a9cf-01d3230a5cb8\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.349021 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.348971    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:56:55.446254 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.446212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:56:55.472697 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.472675    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"]
Jul 29 20:56:55.475258 ip-10-0-139-241 kubenswrapper[2569]: W0729 20:56:55.475234    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e83f419_99fa_4d38_a9cf_01d3230a5cb8.slice/crio-b01a30f8db357ed4a29774c00827eee828f7042da5f2c7526938af7d14aca251 WatchSource:0}: Error finding container b01a30f8db357ed4a29774c00827eee828f7042da5f2c7526938af7d14aca251: Status 404 returned error can't find the container with id b01a30f8db357ed4a29774c00827eee828f7042da5f2c7526938af7d14aca251
Jul 29 20:56:55.476435 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.476418    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:56:55.646763 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:55.646690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7" event={"ID":"3e83f419-99fa-4d38-a9cf-01d3230a5cb8","Type":"ContainerStarted","Data":"b01a30f8db357ed4a29774c00827eee828f7042da5f2c7526938af7d14aca251"}
Jul 29 20:56:56.653482 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:56.653447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7" event={"ID":"3e83f419-99fa-4d38-a9cf-01d3230a5cb8","Type":"ContainerStarted","Data":"8e435099e68ba487756e87ebc75672b96a2109e0ece45cf3850541098dcf9c0e"}
Jul 29 20:56:56.671959 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:56:56.671908    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7" podStartSLOduration=1.5000585100000001 podStartE2EDuration="1.671891459s" totalImagesPullingTime="171.832949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:56:56.669634825 +0000 UTC m=+629.267611182" watchObservedRunningTime="2026-07-29 20:56:56.671891459 +0000 UTC m=+629.269867816"
Jul 29 20:57:05.350101 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:05.350028    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:57:05.350101 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:05.350086    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:57:05.362253 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:05.362232    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:57:05.707305 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:05.707240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7"
Jul 29 20:57:07.151254 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:07.151212    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:57:14.098486 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:14.098455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:57.233832 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:57.233789    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:57.331555 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:57.331528    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:58.300815 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:58.300781    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:57:58.401098 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:57:58.401067    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:26.950258 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:26.950180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:58:36.652252 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:36.652215    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:58:43.460991 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:43.460940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:47.666019 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:47.665984    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 20:58:54.691270 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:54.691197    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:57.946049 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:57.946007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 20:58:58.046593 ip-10-0-139-241 kubenswrapper[2569]: I0729 20:58:58.046563    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:04.720579 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:00:04.720510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:05.938679 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:00:05.938642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:00:32.283168 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:00:32.283132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:01:11.722704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:11.722668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:11.822028 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:11.821990    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:11.945304 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:11.945274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:17.248144 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:17.248111    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:17.829367 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:17.829335    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:17.889376 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:17.889347    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:17.958729 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:17.958702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:18.057119 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:18.057095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:18.161366 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:18.161306    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:23.785806 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:23.785673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:23.932914 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:23.932887    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:01:40.850143 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:40.850098    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:01:50.547138 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:01:50.547101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:03.593276 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:03.593244    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:04.815672 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:04.815638    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:02:05.548635 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:05.548599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:14.754255 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:14.754219    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:24.755447 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:24.755409    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:34.547592 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:34.547553    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:45.350939 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:45.350906    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:02:55.047603 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:02:55.047522    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:03:20.552904 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:03:20.552850    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:03:31.951793 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:03:31.951756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:03:47.173520 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:03:47.173485    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:03:53.957373 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:03:53.957337    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:04:16.352025 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:04:16.351990    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:04:30.750282 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:04:30.750193    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:04:51.950829 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:04:51.950791    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:05:07.250887 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:05:07.250855    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:05:17.154962 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:05:17.154927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:05:26.854002 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:05:26.853972    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:05:36.553893 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:05:36.553858    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:05:45.259817 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:05:45.259783    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:06:06.052528 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:06:06.052447    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:06:16.356151 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:06:16.356110    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:06:25.949879 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:06:25.949839    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:08:36.256460 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:08:36.256422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:08:43.755409 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:08:43.755370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:05.955906 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:05.955825    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:17.561024 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:17.560987    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:26.544483 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:26.544445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:09:28.352041 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:28.352007    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:37.351129 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:37.351090    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:48.451380 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:48.451344    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:09:59.979851 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:09:59.979819    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:10:00.048823 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:00.048791    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:10:08.148132 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:08.148102    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:10:18.955536 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:18.955495    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:10:30.049644 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:30.049560    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:10:37.048097 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:37.048063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:10:57.761590 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:10:57.761548    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:11:07.255788 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:07.255749    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:11:10.550591 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.550553    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:10.554437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.554413    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.557155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.557117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e53236bf-kserve-self-signed-certs\""
Jul 29 21:11:10.569608 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.569564    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:10.623346 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.623447 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623359    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.623447 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623426    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.623521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.623521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623507    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.623586 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.623521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724134 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724233 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724233 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724309 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724309 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724309 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724305    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724582 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724554    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724582 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.724776 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.724652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.726502 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.726481    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.726691 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.726669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.731671 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.731648    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw\") pod \"e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") " pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.864252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.864202    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:10.991287 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:10.991258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:10.994361 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:11:10.994335    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddedead49_d55b_4c0e_b29d_ef6e05ca7ea0.slice/crio-a68cdfa8c618b836eb3b799e5d8eff0fd3277fc0f686f71824db74582cf86f43 WatchSource:0}: Error finding container a68cdfa8c618b836eb3b799e5d8eff0fd3277fc0f686f71824db74582cf86f43: Status 404 returned error can't find the container with id a68cdfa8c618b836eb3b799e5d8eff0fd3277fc0f686f71824db74582cf86f43
Jul 29 21:11:11.414471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:11.414440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" event={"ID":"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0","Type":"ContainerStarted","Data":"afa4c42037f35f5ca621e726da2dc046ae307698693c4ac9e0bfd0f8a71dbccc"}
Jul 29 21:11:11.414648 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:11.414479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" event={"ID":"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0","Type":"ContainerStarted","Data":"a68cdfa8c618b836eb3b799e5d8eff0fd3277fc0f686f71824db74582cf86f43"}
Jul 29 21:11:11.430457 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:11.430410    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" podStartSLOduration=1.430394609 podStartE2EDuration="1.430394609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:11:11.429475327 +0000 UTC m=+1484.027451685" watchObservedRunningTime="2026-07-29 21:11:11.430394609 +0000 UTC m=+1484.028370990"
Jul 29 21:11:11.453373 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:11.453345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:14.967287 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:14.967245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:20.864790 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:20.864698    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:20.864790 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:20.864765    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:20.877036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:20.877013    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:21.469900 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:21.469873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:21.742864 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:21.742832    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:11:40.388710 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:40.388675    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:40.412054 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:40.412027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:41.277198 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:41.277062    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:11:44.452282 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:44.452247    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:11:55.408358 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.408269    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" podUID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" containerName="main" containerID="cri-o://afa4c42037f35f5ca621e726da2dc046ae307698693c4ac9e0bfd0f8a71dbccc" gracePeriod=45
Jul 29 21:11:55.625171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.625134    2569 generic.go:356] "Generic (PLEG): container finished" podID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" containerID="afa4c42037f35f5ca621e726da2dc046ae307698693c4ac9e0bfd0f8a71dbccc" exitCode=0
Jul 29 21:11:55.625297 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.625204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" event={"ID":"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0","Type":"ContainerDied","Data":"afa4c42037f35f5ca621e726da2dc046ae307698693c4ac9e0bfd0f8a71dbccc"}
Jul 29 21:11:55.654341 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.654318    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:55.779596 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779575    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.779755 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779607    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.779755 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779656    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.779755 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779684    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.779755 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779708    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.779974 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779765    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw\") pod \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\" (UID: \"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0\") "
Jul 29 21:11:55.780037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779978    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache" (OuterVolumeSpecName: "model-cache") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:11:55.780037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.779997    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:11:55.780037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.780009    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home" (OuterVolumeSpecName: "home") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:11:55.781856 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.781828    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:11:55.781945 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.781860    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm" (OuterVolumeSpecName: "dshm") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:11:55.781986 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.781963    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw" (OuterVolumeSpecName: "kube-api-access-7zlnw") pod "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" (UID: "dedead49-d55b-4c0e-b29d-ef6e05ca7ea0"). InnerVolumeSpecName "kube-api-access-7zlnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:11:55.881037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881005    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tmp-dir\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:55.881037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881038    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zlnw\" (UniqueName: \"kubernetes.io/projected/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-kube-api-access-7zlnw\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:55.881171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881049    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-tls-certs\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:55.881171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881057    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-home\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:55.881171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881065    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-dshm\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:55.881171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:55.881073    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0-model-cache\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:11:56.632298 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:56.632265    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"
Jul 29 21:11:56.632298 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:56.632285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p" event={"ID":"dedead49-d55b-4c0e-b29d-ef6e05ca7ea0","Type":"ContainerDied","Data":"a68cdfa8c618b836eb3b799e5d8eff0fd3277fc0f686f71824db74582cf86f43"}
Jul 29 21:11:56.632796 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:56.632338    2569 scope.go:117] "RemoveContainer" containerID="afa4c42037f35f5ca621e726da2dc046ae307698693c4ac9e0bfd0f8a71dbccc"
Jul 29 21:11:56.648374 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:56.648346    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:56.653608 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:56.653582    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e53236bf-kserve-8565c8777f-g277p"]
Jul 29 21:11:57.897774 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:57.897725    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" path="/var/lib/kubelet/pods/dedead49-d55b-4c0e-b29d-ef6e05ca7ea0/volumes"
Jul 29 21:11:58.453389 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:11:58.453350    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:05.255446 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:05.255410    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:10.711263 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:10.711230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:12:14.753907 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:14.753865    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:21.549949 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:21.549913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:35.059359 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:35.059318    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:46.045867 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:46.045829    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:12:56.059686 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:12:56.059654    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:05.360714 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:05.360676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:15.161100 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:15.161062    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:25.753924 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:25.753838    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:42.573870 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:42.573832    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:52.359572 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:52.359542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:13:59.352594 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:13:59.352563    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:14:08.752565 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:14:08.752533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:14:15.354762 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:14:15.353702    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:14:27.650017 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:14:27.649979    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:14:42.549359 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:14:42.549328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:14:55.354525 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:14:55.354452    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:15:10.158817 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:15:10.158789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:15:25.249639 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:15:25.249607    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:15:34.650251 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:15:34.650219    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:15:36.534991 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:15:36.534953    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:15:47.847494 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:15:47.847460    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:16:03.061017 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:03.060982    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:16:12.749476 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:12.749441    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:16:13.155499 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:13.155408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:16.383526 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:16.383479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:16.483792 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:16.483717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:16.894703 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:16.894668    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:22.552298 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:22.552224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:16:22.721340 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:22.721304    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:22.819978 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:22.819910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:22.999929 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:22.999899    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:23.290307 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:23.290276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:24.281339 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:24.281296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:25.642189 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:25.642155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:25.745976 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:25.745938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:26.823795 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:26.823763    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:26.925004 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:26.924979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:27.268923 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.268886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:27.363598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.363552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:27.364373 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.364351    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:27.364700 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.364687    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" containerName="main"
Jul 29 21:16:27.364758 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.364702    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" containerName="main"
Jul 29 21:16:27.364794 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.364782    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="dedead49-d55b-4c0e-b29d-ef6e05ca7ea0" containerName="main"
Jul 29 21:16:27.367703 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.367688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.370068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.370047    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-23c87980\""
Jul 29 21:16:27.370171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.370047    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-23c87980-dockercfg-qmv58\""
Jul 29 21:16:27.377358 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.377336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:27.500080 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.500034    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlr4d\" (UniqueName: \"kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.500220 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.500091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.601328 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.601259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlr4d\" (UniqueName: \"kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.601328 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.601311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.602022 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.602000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.609844 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.609822    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlr4d\" (UniqueName: \"kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d\") pod \"payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.678471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.678453    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:27.815624 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.815589    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:27.819653 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:16:27.819624    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bc07c1_e107_4c58_8bfa_183a4f47578d.slice/crio-1fbc8e331c9ce3abc53db9d9051105b04bfd780325aea228d56b5766797cce1d WatchSource:0}: Error finding container 1fbc8e331c9ce3abc53db9d9051105b04bfd780325aea228d56b5766797cce1d: Status 404 returned error can't find the container with id 1fbc8e331c9ce3abc53db9d9051105b04bfd780325aea228d56b5766797cce1d
Jul 29 21:16:27.821959 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:27.821939    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:16:28.821126 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:28.821086    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" event={"ID":"e6bc07c1-e107-4c58-8bfa-183a4f47578d","Type":"ContainerStarted","Data":"1fbc8e331c9ce3abc53db9d9051105b04bfd780325aea228d56b5766797cce1d"}
Jul 29 21:16:30.832661 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:30.832626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" event={"ID":"e6bc07c1-e107-4c58-8bfa-183a4f47578d","Type":"ContainerStarted","Data":"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"}
Jul 29 21:16:30.833041 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:30.832802    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:30.849659 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:30.849620    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" podStartSLOduration=1.69181334 podStartE2EDuration="3.849607233s" totalImagesPullingTime="2.157793893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:30.849061885 +0000 UTC m=+1803.447038261" watchObservedRunningTime="2026-07-29 21:16:30.849607233 +0000 UTC m=+1803.447583567"
Jul 29 21:16:33.251660 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:33.251631    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:16:41.839689 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:41.839657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:57.215450 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.215418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.250580 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.250546    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:57.250799 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.250773    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" podUID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" containerName="payload-pre-processing" containerID="cri-o://ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a" gracePeriod=30
Jul 29 21:16:57.275240 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.275211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.335545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.335520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.429533 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.429501    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.497635 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.497612    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:57.593500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.593466    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.644428 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.644393    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlr4d\" (UniqueName: \"kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d\") pod \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") "
Jul 29 21:16:57.644539 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.644516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume\") pod \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\" (UID: \"e6bc07c1-e107-4c58-8bfa-183a4f47578d\") "
Jul 29 21:16:57.644903 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.644884    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6bc07c1-e107-4c58-8bfa-183a4f47578d" (UID: "e6bc07c1-e107-4c58-8bfa-183a4f47578d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:16:57.646444 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.646425    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d" (OuterVolumeSpecName: "kube-api-access-mlr4d") pod "e6bc07c1-e107-4c58-8bfa-183a4f47578d" (UID: "e6bc07c1-e107-4c58-8bfa-183a4f47578d"). InnerVolumeSpecName "kube-api-access-mlr4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:16:57.745417 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.745348    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6bc07c1-e107-4c58-8bfa-183a4f47578d-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:16:57.745417 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.745384    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlr4d\" (UniqueName: \"kubernetes.io/projected/e6bc07c1-e107-4c58-8bfa-183a4f47578d-kube-api-access-mlr4d\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:16:57.831125 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.831101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:57.961100 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.961067    2569 generic.go:356] "Generic (PLEG): container finished" podID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" containerID="ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a" exitCode=0
Jul 29 21:16:57.961214 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.961123    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"
Jul 29 21:16:57.961214 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.961147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" event={"ID":"e6bc07c1-e107-4c58-8bfa-183a4f47578d","Type":"ContainerDied","Data":"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"}
Jul 29 21:16:57.961289 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.961220    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg" event={"ID":"e6bc07c1-e107-4c58-8bfa-183a4f47578d","Type":"ContainerDied","Data":"1fbc8e331c9ce3abc53db9d9051105b04bfd780325aea228d56b5766797cce1d"}
Jul 29 21:16:57.961289 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.961242    2569 scope.go:117] "RemoveContainer" containerID="ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"
Jul 29 21:16:57.969520 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.969502    2569 scope.go:117] "RemoveContainer" containerID="ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"
Jul 29 21:16:57.969777 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:16:57.969756    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a\": container with ID starting with ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a not found: ID does not exist" containerID="ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"
Jul 29 21:16:57.969843 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.969788    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a"} err="failed to get container status \"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a\": rpc error: code = NotFound desc = could not find container \"ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a\": container with ID starting with ec76b7f437b017359f165f1cd71b96047bffcce6069a7c02dcb469410561427a not found: ID does not exist"
Jul 29 21:16:57.976611 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.976588    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:57.978437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:57.978419    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-23c87980-d9bcf7688-rltvg"]
Jul 29 21:16:58.068675 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:58.068612    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:58.168534 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:58.168502    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:58.268979 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:58.268941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:58.605060 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:58.605031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:58.937935 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:58.937858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:16:59.897085 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:16:59.897051    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" path="/var/lib/kubelet/pods/e6bc07c1-e107-4c58-8bfa-183a4f47578d/volumes"
Jul 29 21:17:01.023553 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.023519    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:01.124038 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.124010    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:01.162181 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.162150    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:17:01.223486 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.223456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:01.534568 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.534529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:01.639846 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.639811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:01.646900 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.646878    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:01.647208 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.647196    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" containerName="payload-pre-processing"
Jul 29 21:17:01.647249 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.647209    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" containerName="payload-pre-processing"
Jul 29 21:17:01.647284 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.647269    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6bc07c1-e107-4c58-8bfa-183a4f47578d" containerName="payload-pre-processing"
Jul 29 21:17:01.650044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.650029    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.652750 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.652712    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-946d3187\""
Jul 29 21:17:01.652848 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.652763    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-946d3187-dockercfg-6z9zx\""
Jul 29 21:17:01.660235 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.660211    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:01.667858 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.667837    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:01.670838 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.670824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.678651 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.678629    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxv2\" (UniqueName: \"kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.678770 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.678688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.678770 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.678757    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gmqr\" (UniqueName: \"kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.678909 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.678781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.686302 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.686281    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:01.779389 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.779367    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxv2\" (UniqueName: \"kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.779502 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.779414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.779502 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.779446    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gmqr\" (UniqueName: \"kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.779502 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.779463    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.780096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.780080    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.780196 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.780180    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.787709 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.787655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxv2\" (UniqueName: \"kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2\") pod \"payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") " pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:01.787989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.787970    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gmqr\" (UniqueName: \"kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr\") pod \"payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.961029 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.961004    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:01.979909 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:01.979886    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:02.102139 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.101931    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:02.106210 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:17:02.106170    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac6d8eb_7d14_442a_8cad_1d0e24c0abfd.slice/crio-fc573baf6971cbc98e0e66798f49b4837fa1f6160ae903d3400032614bcaacc4 WatchSource:0}: Error finding container fc573baf6971cbc98e0e66798f49b4837fa1f6160ae903d3400032614bcaacc4: Status 404 returned error can't find the container with id fc573baf6971cbc98e0e66798f49b4837fa1f6160ae903d3400032614bcaacc4
Jul 29 21:17:02.328934 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.328910    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:02.331162 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:17:02.331135    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef5922a_6438_4a89_a636_46befbf7d90e.slice/crio-11a0df48e0386377b2c1e7a04b463219db41c292d9c49f4edb5ddc348ff66761 WatchSource:0}: Error finding container 11a0df48e0386377b2c1e7a04b463219db41c292d9c49f4edb5ddc348ff66761: Status 404 returned error can't find the container with id 11a0df48e0386377b2c1e7a04b463219db41c292d9c49f4edb5ddc348ff66761
Jul 29 21:17:02.984291 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.984255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" event={"ID":"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd","Type":"ContainerStarted","Data":"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"}
Jul 29 21:17:02.984577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.984335    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:02.984577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.984363    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" event={"ID":"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd","Type":"ContainerStarted","Data":"fc573baf6971cbc98e0e66798f49b4837fa1f6160ae903d3400032614bcaacc4"}
Jul 29 21:17:02.985886 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.985860    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" event={"ID":"1ef5922a-6438-4a89-a636-46befbf7d90e","Type":"ContainerStarted","Data":"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"}
Jul 29 21:17:02.985993 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.985900    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" event={"ID":"1ef5922a-6438-4a89-a636-46befbf7d90e","Type":"ContainerStarted","Data":"11a0df48e0386377b2c1e7a04b463219db41c292d9c49f4edb5ddc348ff66761"}
Jul 29 21:17:02.985993 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:02.985945    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:03.001101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:03.001057    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" podStartSLOduration=2.001041673 podStartE2EDuration="2.001041673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:02.999718293 +0000 UTC m=+1835.597694676" watchObservedRunningTime="2026-07-29 21:17:03.001041673 +0000 UTC m=+1835.599018044"
Jul 29 21:17:03.017182 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:03.017122    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" podStartSLOduration=2.01710449 podStartE2EDuration="2.01710449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:03.014901019 +0000 UTC m=+1835.612877377" watchObservedRunningTime="2026-07-29 21:17:03.01710449 +0000 UTC m=+1835.615080848"
Jul 29 21:17:11.354541 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:11.354510    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:17:13.991902 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:13.991875    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:13.992264 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:13.991943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:32.394139 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.394103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:32.420290 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.420260    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:32.420466 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.420446    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" podUID="1ef5922a-6438-4a89-a636-46befbf7d90e" containerName="payload-processing" containerID="cri-o://ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1" gracePeriod=30
Jul 29 21:17:32.433750 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.433702    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:32.434082 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.434051    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" podUID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" containerName="payload-pre-processing" containerID="cri-o://ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f" gracePeriod=30
Jul 29 21:17:32.463634 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.463605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:32.528397 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.528366    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:32.628864 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.628831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:32.700247 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.700224    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:32.704008 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.703989    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:32.728194 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.728159    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:32.808507 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.808480    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxv2\" (UniqueName: \"kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2\") pod \"1ef5922a-6438-4a89-a636-46befbf7d90e\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") "
Jul 29 21:17:32.808610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.808512    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume\") pod \"1ef5922a-6438-4a89-a636-46befbf7d90e\" (UID: \"1ef5922a-6438-4a89-a636-46befbf7d90e\") "
Jul 29 21:17:32.808610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.808553    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gmqr\" (UniqueName: \"kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr\") pod \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") "
Jul 29 21:17:32.808610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.808595    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume\") pod \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\" (UID: \"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd\") "
Jul 29 21:17:32.809014 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.808988    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ef5922a-6438-4a89-a636-46befbf7d90e" (UID: "1ef5922a-6438-4a89-a636-46befbf7d90e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:17:32.809141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.809054    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" (UID: "eac6d8eb-7d14-442a-8cad-1d0e24c0abfd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:17:32.810598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.810580    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2" (OuterVolumeSpecName: "kube-api-access-gxxv2") pod "1ef5922a-6438-4a89-a636-46befbf7d90e" (UID: "1ef5922a-6438-4a89-a636-46befbf7d90e"). InnerVolumeSpecName "kube-api-access-gxxv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:17:32.810663 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.810649    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr" (OuterVolumeSpecName: "kube-api-access-6gmqr") pod "eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" (UID: "eac6d8eb-7d14-442a-8cad-1d0e24c0abfd"). InnerVolumeSpecName "kube-api-access-6gmqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:17:32.909856 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.909833    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxv2\" (UniqueName: \"kubernetes.io/projected/1ef5922a-6438-4a89-a636-46befbf7d90e-kube-api-access-gxxv2\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:17:32.909856 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.909854    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ef5922a-6438-4a89-a636-46befbf7d90e-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:17:32.909983 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.909866    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gmqr\" (UniqueName: \"kubernetes.io/projected/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-kube-api-access-6gmqr\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:17:32.909983 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.909875    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:17:32.941958 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:32.941896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:33.119650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.119624    2569 generic.go:356] "Generic (PLEG): container finished" podID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" containerID="ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f" exitCode=0
Jul 29 21:17:33.119771 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.119664    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"
Jul 29 21:17:33.119771 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.119710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" event={"ID":"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd","Type":"ContainerDied","Data":"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"}
Jul 29 21:17:33.119891 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.119780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr" event={"ID":"eac6d8eb-7d14-442a-8cad-1d0e24c0abfd","Type":"ContainerDied","Data":"fc573baf6971cbc98e0e66798f49b4837fa1f6160ae903d3400032614bcaacc4"}
Jul 29 21:17:33.119891 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.119814    2569 scope.go:117] "RemoveContainer" containerID="ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"
Jul 29 21:17:33.121075 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.121056    2569 generic.go:356] "Generic (PLEG): container finished" podID="1ef5922a-6438-4a89-a636-46befbf7d90e" containerID="ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1" exitCode=0
Jul 29 21:17:33.121161 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.121085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" event={"ID":"1ef5922a-6438-4a89-a636-46befbf7d90e","Type":"ContainerDied","Data":"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"}
Jul 29 21:17:33.121161 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.121104    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"
Jul 29 21:17:33.121161 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.121130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt" event={"ID":"1ef5922a-6438-4a89-a636-46befbf7d90e","Type":"ContainerDied","Data":"11a0df48e0386377b2c1e7a04b463219db41c292d9c49f4edb5ddc348ff66761"}
Jul 29 21:17:33.130109 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.130090    2569 scope.go:117] "RemoveContainer" containerID="ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"
Jul 29 21:17:33.130384 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:17:33.130365    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f\": container with ID starting with ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f not found: ID does not exist" containerID="ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"
Jul 29 21:17:33.130471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.130388    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f"} err="failed to get container status \"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f\": rpc error: code = NotFound desc = could not find container \"ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f\": container with ID starting with ecaf9e0f0744f539fb89a9b4c7fc2c5b4f568c04848e106cb502980c48223e3f not found: ID does not exist"
Jul 29 21:17:33.130471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.130404    2569 scope.go:117] "RemoveContainer" containerID="ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"
Jul 29 21:17:33.137822 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.137808    2569 scope.go:117] "RemoveContainer" containerID="ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"
Jul 29 21:17:33.138062 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:17:33.138046    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1\": container with ID starting with ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1 not found: ID does not exist" containerID="ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"
Jul 29 21:17:33.138111 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.138065    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1"} err="failed to get container status \"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1\": rpc error: code = NotFound desc = could not find container \"ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1\": container with ID starting with ec7d70ff92a66bb7a5bc51fde53ac2ac916e143ab4c995db9154e0d890adf3c1 not found: ID does not exist"
Jul 29 21:17:33.144851 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.144831    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:33.148691 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.148673    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-946d3187-648fb8bfb4-9rtpr"]
Jul 29 21:17:33.160801 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.160782    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:33.164958 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.164941    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-946d3187-6fbb98bb9c-qfrkt"]
Jul 29 21:17:33.211157 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.208070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:33.859141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.859112    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:33.897056 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.897030    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef5922a-6438-4a89-a636-46befbf7d90e" path="/var/lib/kubelet/pods/1ef5922a-6438-4a89-a636-46befbf7d90e/volumes"
Jul 29 21:17:33.897381 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.897369    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" path="/var/lib/kubelet/pods/eac6d8eb-7d14-442a-8cad-1d0e24c0abfd/volumes"
Jul 29 21:17:33.918699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:33.918667    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.344379 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:36.344351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.445148 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:36.445118    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.466391 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:36.466367    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:17:37.211702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.211665    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.392011 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.391974    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.761482 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.761454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.859508 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.859469    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.869445 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869418    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:17:37.869772 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869758    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef5922a-6438-4a89-a636-46befbf7d90e" containerName="payload-processing"
Jul 29 21:17:37.869831 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869773    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef5922a-6438-4a89-a636-46befbf7d90e" containerName="payload-processing"
Jul 29 21:17:37.869831 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869788    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" containerName="payload-pre-processing"
Jul 29 21:17:37.869831 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869794    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" containerName="payload-pre-processing"
Jul 29 21:17:37.869934 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869866    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac6d8eb-7d14-442a-8cad-1d0e24c0abfd" containerName="payload-pre-processing"
Jul 29 21:17:37.869934 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.869880    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef5922a-6438-4a89-a636-46befbf7d90e" containerName="payload-processing"
Jul 29 21:17:37.872835 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.872814    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:37.875929 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.875903    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-17bc2b8b\""
Jul 29 21:17:37.876078 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.875997    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-17bc2b8b-dockercfg-rvgqb\""
Jul 29 21:17:37.891226 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.891204    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:17:37.936629 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.936604    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:17:37.938547 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.938533    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:37.952444 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.952423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:37.952547 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.952469    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5mr9\" (UniqueName: \"kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:37.958643 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:37.958621    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:17:38.053538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.053487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6chn\" (UniqueName: \"kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.053538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.053516    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.053665 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.053542    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5mr9\" (UniqueName: \"kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:38.053665 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.053638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:38.054214 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.054198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:38.066846 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.066818    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5mr9\" (UniqueName: \"kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9\") pod \"payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:38.154533 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.154504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6chn\" (UniqueName: \"kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.154649 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.154544    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.155252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.155235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.162014 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.161997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6chn\" (UniqueName: \"kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn\") pod \"payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") " pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.182507 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.182486    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:38.247558 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.247532    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:38.317345 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.317316    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:17:38.320969 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:17:38.320929    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3861de9_145b_497f_8625_a3d3b24bee1b.slice/crio-bbbe6c0db02ad7378ccc1df43e60cb3d18e99913517b659a1e48f07fa9cb6635 WatchSource:0}: Error finding container bbbe6c0db02ad7378ccc1df43e60cb3d18e99913517b659a1e48f07fa9cb6635: Status 404 returned error can't find the container with id bbbe6c0db02ad7378ccc1df43e60cb3d18e99913517b659a1e48f07fa9cb6635
Jul 29 21:17:38.387061 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:38.386996    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:17:38.392172 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:17:38.392147    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bac2189_048b_43cd_8a80_a6e778ec7bce.slice/crio-1575d00c0bc41ee089d51727b0ae34fb3637ba9fd1d16416ab54ee618feca525 WatchSource:0}: Error finding container 1575d00c0bc41ee089d51727b0ae34fb3637ba9fd1d16416ab54ee618feca525: Status 404 returned error can't find the container with id 1575d00c0bc41ee089d51727b0ae34fb3637ba9fd1d16416ab54ee618feca525
Jul 29 21:17:39.151252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.151213    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" event={"ID":"e3861de9-145b-497f-8625-a3d3b24bee1b","Type":"ContainerStarted","Data":"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"}
Jul 29 21:17:39.151412 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.151280    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:39.151412 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.151305    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" event={"ID":"e3861de9-145b-497f-8625-a3d3b24bee1b","Type":"ContainerStarted","Data":"bbbe6c0db02ad7378ccc1df43e60cb3d18e99913517b659a1e48f07fa9cb6635"}
Jul 29 21:17:39.152639 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.152615    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" event={"ID":"7bac2189-048b-43cd-8a80-a6e778ec7bce","Type":"ContainerStarted","Data":"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"}
Jul 29 21:17:39.152772 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.152656    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" event={"ID":"7bac2189-048b-43cd-8a80-a6e778ec7bce","Type":"ContainerStarted","Data":"1575d00c0bc41ee089d51727b0ae34fb3637ba9fd1d16416ab54ee618feca525"}
Jul 29 21:17:39.152772 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.152754    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:17:39.168530 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.168488    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" podStartSLOduration=2.168476611 podStartE2EDuration="2.168476611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:39.166813611 +0000 UTC m=+1871.764789966" watchObservedRunningTime="2026-07-29 21:17:39.168476611 +0000 UTC m=+1871.766452967"
Jul 29 21:17:39.186010 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:39.185961    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" podStartSLOduration=2.185945312 podStartE2EDuration="2.185945312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:39.183111773 +0000 UTC m=+1871.781088130" watchObservedRunningTime="2026-07-29 21:17:39.185945312 +0000 UTC m=+1871.783921671"
Jul 29 21:17:46.653228 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:46.653195    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:17:50.158994 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:50.158904    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:17:50.158994 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:17:50.158989    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:18:03.414979 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:03.414939    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:03.468262 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:03.468232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:07.753296 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.753254    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:07.781687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.781660    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:18:07.781932 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.781904    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" podUID="7bac2189-048b-43cd-8a80-a6e778ec7bce" containerName="payload-processing" containerID="cri-o://a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda" gracePeriod=30
Jul 29 21:18:07.788538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.788509    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:18:07.788753 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.788715    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" podUID="e3861de9-145b-497f-8625-a3d3b24bee1b" containerName="payload-pre-processing" containerID="cri-o://1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c" gracePeriod=30
Jul 29 21:18:07.819499 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.819459    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:07.929948 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:07.929920    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.054015 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.053989    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:18:08.068445 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.068426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:18:08.074913 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.074884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.189952 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.189926    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume\") pod \"7bac2189-048b-43cd-8a80-a6e778ec7bce\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") "
Jul 29 21:18:08.190091 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.189992    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume\") pod \"e3861de9-145b-497f-8625-a3d3b24bee1b\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") "
Jul 29 21:18:08.190091 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.190012    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5mr9\" (UniqueName: \"kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9\") pod \"e3861de9-145b-497f-8625-a3d3b24bee1b\" (UID: \"e3861de9-145b-497f-8625-a3d3b24bee1b\") "
Jul 29 21:18:08.190091 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.190066    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6chn\" (UniqueName: \"kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn\") pod \"7bac2189-048b-43cd-8a80-a6e778ec7bce\" (UID: \"7bac2189-048b-43cd-8a80-a6e778ec7bce\") "
Jul 29 21:18:08.190330 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.190306    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3861de9-145b-497f-8625-a3d3b24bee1b" (UID: "e3861de9-145b-497f-8625-a3d3b24bee1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:08.190384 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.190322    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7bac2189-048b-43cd-8a80-a6e778ec7bce" (UID: "7bac2189-048b-43cd-8a80-a6e778ec7bce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:08.192141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.192116    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn" (OuterVolumeSpecName: "kube-api-access-d6chn") pod "7bac2189-048b-43cd-8a80-a6e778ec7bce" (UID: "7bac2189-048b-43cd-8a80-a6e778ec7bce"). InnerVolumeSpecName "kube-api-access-d6chn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:08.192214 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.192145    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9" (OuterVolumeSpecName: "kube-api-access-f5mr9") pod "e3861de9-145b-497f-8625-a3d3b24bee1b" (UID: "e3861de9-145b-497f-8625-a3d3b24bee1b"). InnerVolumeSpecName "kube-api-access-f5mr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:08.287036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.286982    2569 generic.go:356] "Generic (PLEG): container finished" podID="7bac2189-048b-43cd-8a80-a6e778ec7bce" containerID="a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda" exitCode=0
Jul 29 21:18:08.287036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.287028    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"
Jul 29 21:18:08.287166 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.287070    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" event={"ID":"7bac2189-048b-43cd-8a80-a6e778ec7bce","Type":"ContainerDied","Data":"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"}
Jul 29 21:18:08.287166 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.287123    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l" event={"ID":"7bac2189-048b-43cd-8a80-a6e778ec7bce","Type":"ContainerDied","Data":"1575d00c0bc41ee089d51727b0ae34fb3637ba9fd1d16416ab54ee618feca525"}
Jul 29 21:18:08.287166 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.287152    2569 scope.go:117] "RemoveContainer" containerID="a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"
Jul 29 21:18:08.288280 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.288261    2569 generic.go:356] "Generic (PLEG): container finished" podID="e3861de9-145b-497f-8625-a3d3b24bee1b" containerID="1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c" exitCode=0
Jul 29 21:18:08.288363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.288304    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"
Jul 29 21:18:08.288363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.288313    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" event={"ID":"e3861de9-145b-497f-8625-a3d3b24bee1b","Type":"ContainerDied","Data":"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"}
Jul 29 21:18:08.288363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.288346    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk" event={"ID":"e3861de9-145b-497f-8625-a3d3b24bee1b","Type":"ContainerDied","Data":"bbbe6c0db02ad7378ccc1df43e60cb3d18e99913517b659a1e48f07fa9cb6635"}
Jul 29 21:18:08.290637 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.290609    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3861de9-145b-497f-8625-a3d3b24bee1b-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:08.290753 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.290639    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5mr9\" (UniqueName: \"kubernetes.io/projected/e3861de9-145b-497f-8625-a3d3b24bee1b-kube-api-access-f5mr9\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:08.290753 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.290654    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6chn\" (UniqueName: \"kubernetes.io/projected/7bac2189-048b-43cd-8a80-a6e778ec7bce-kube-api-access-d6chn\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:08.290753 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.290668    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7bac2189-048b-43cd-8a80-a6e778ec7bce-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:08.295601 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.295586    2569 scope.go:117] "RemoveContainer" containerID="a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"
Jul 29 21:18:08.295904 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:18:08.295882    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda\": container with ID starting with a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda not found: ID does not exist" containerID="a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"
Jul 29 21:18:08.296010 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.295908    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda"} err="failed to get container status \"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda\": rpc error: code = NotFound desc = could not find container \"a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda\": container with ID starting with a90867389406247183966e0714d9df26d0e524fa4e62c138221e94db8c34dfda not found: ID does not exist"
Jul 29 21:18:08.296010 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.295924    2569 scope.go:117] "RemoveContainer" containerID="1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"
Jul 29 21:18:08.303631 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.303616    2569 scope.go:117] "RemoveContainer" containerID="1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"
Jul 29 21:18:08.303898 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:18:08.303854    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c\": container with ID starting with 1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c not found: ID does not exist" containerID="1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"
Jul 29 21:18:08.303984 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.303904    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c"} err="failed to get container status \"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c\": rpc error: code = NotFound desc = could not find container \"1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c\": container with ID starting with 1a19d93e5b75428893df617fe95976c1c8f1ac8c99be281048be8a3dbb11052c not found: ID does not exist"
Jul 29 21:18:08.311487 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.311463    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:18:08.313208 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.313188    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-17bc2b8b-76d4f6d46d-bsvlk"]
Jul 29 21:18:08.323255 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.323226    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:18:08.326706 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:08.326686    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-17bc2b8b-85656d6794-4qn9l"]
Jul 29 21:18:09.099693 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.099616    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.361113 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.361041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.538399 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.538374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.838046 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.838017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:09.897044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.897017    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bac2189-048b-43cd-8a80-a6e778ec7bce" path="/var/lib/kubelet/pods/7bac2189-048b-43cd-8a80-a6e778ec7bce/volumes"
Jul 29 21:18:09.897361 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:09.897349    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3861de9-145b-497f-8625-a3d3b24bee1b" path="/var/lib/kubelet/pods/e3861de9-145b-497f-8625-a3d3b24bee1b/volumes"
Jul 29 21:18:10.002178 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:10.002151    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:11.352391 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:11.352359    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:18:12.158096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:12.158044    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.260069 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:12.260035    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:13.376382 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:13.376345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:13.665630 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:13.665551    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:14.004341 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004311    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:14.004655 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004642    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bac2189-048b-43cd-8a80-a6e778ec7bce" containerName="payload-processing"
Jul 29 21:18:14.004699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004656    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bac2189-048b-43cd-8a80-a6e778ec7bce" containerName="payload-processing"
Jul 29 21:18:14.004699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004668    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3861de9-145b-497f-8625-a3d3b24bee1b" containerName="payload-pre-processing"
Jul 29 21:18:14.004699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004673    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3861de9-145b-497f-8625-a3d3b24bee1b" containerName="payload-pre-processing"
Jul 29 21:18:14.004824 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004759    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3861de9-145b-497f-8625-a3d3b24bee1b" containerName="payload-pre-processing"
Jul 29 21:18:14.004824 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.004768    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bac2189-048b-43cd-8a80-a6e778ec7bce" containerName="payload-processing"
Jul 29 21:18:14.007450 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.007436    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.010307 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.010288    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2d0e6b56\""
Jul 29 21:18:14.031122 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.031101    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tndf7\" (UniqueName: \"kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.031223 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.031171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.036553 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.036533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:14.052334 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.052310    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:14.054555 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.054539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.059261 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.059242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2d0e6b56-dockercfg-mcxqf\""
Jul 29 21:18:14.074527 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.074505    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:14.131562 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.131541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.131658 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.131579    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.131658 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.131641    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7xh\" (UniqueName: \"kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.131761 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.131673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tndf7\" (UniqueName: \"kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.132266 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.132248    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.133869 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.133843    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:14.139927 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.139910    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tndf7\" (UniqueName: \"kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7\") pod \"payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") " pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.232250 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.232219    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.232342 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.232277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7xh\" (UniqueName: \"kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.233141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.233114    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:14.233242 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.233140    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.240835 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.240816    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7xh\" (UniqueName: \"kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh\") pod \"payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.316618 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.316566    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:14.366108 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.365218    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:14.656156 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.656127    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:14.658809 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:18:14.658780    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39bb01f_910f_4426_820a_67df6cacbc1e.slice/crio-0c7f63014a94f2143df7c6373d21e690749ae2169d6b0cd5218a71932d1227fd WatchSource:0}: Error finding container 0c7f63014a94f2143df7c6373d21e690749ae2169d6b0cd5218a71932d1227fd: Status 404 returned error can't find the container with id 0c7f63014a94f2143df7c6373d21e690749ae2169d6b0cd5218a71932d1227fd
Jul 29 21:18:14.710356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:14.710333    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:14.713127 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:18:14.713104    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f8401f_5ec4_4faa_a4ab_fb89330e827e.slice/crio-a0c359dd8d3a0adde625116077970fec97b0f2f33a4485d12641ffe15b23db3b WatchSource:0}: Error finding container a0c359dd8d3a0adde625116077970fec97b0f2f33a4485d12641ffe15b23db3b: Status 404 returned error can't find the container with id a0c359dd8d3a0adde625116077970fec97b0f2f33a4485d12641ffe15b23db3b
Jul 29 21:18:15.322675 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.322636    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" event={"ID":"f39bb01f-910f-4426-820a-67df6cacbc1e","Type":"ContainerStarted","Data":"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"}
Jul 29 21:18:15.322891 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.322692    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:15.322891 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.322718    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" event={"ID":"f39bb01f-910f-4426-820a-67df6cacbc1e","Type":"ContainerStarted","Data":"0c7f63014a94f2143df7c6373d21e690749ae2169d6b0cd5218a71932d1227fd"}
Jul 29 21:18:15.324120 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.324093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" event={"ID":"d6f8401f-5ec4-4faa-a4ab-fb89330e827e","Type":"ContainerStarted","Data":"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"}
Jul 29 21:18:15.324229 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.324141    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" event={"ID":"d6f8401f-5ec4-4faa-a4ab-fb89330e827e","Type":"ContainerStarted","Data":"a0c359dd8d3a0adde625116077970fec97b0f2f33a4485d12641ffe15b23db3b"}
Jul 29 21:18:15.324229 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.324181    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:15.340465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.340421    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" podStartSLOduration=2.340406238 podStartE2EDuration="2.340406238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:15.339409792 +0000 UTC m=+1907.937386168" watchObservedRunningTime="2026-07-29 21:18:15.340406238 +0000 UTC m=+1907.938382635"
Jul 29 21:18:15.354889 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:15.354845    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" podStartSLOduration=1.354833875 podStartE2EDuration="1.354833875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:15.352789331 +0000 UTC m=+1907.950765687" watchObservedRunningTime="2026-07-29 21:18:15.354833875 +0000 UTC m=+1907.952810231"
Jul 29 21:18:22.157443 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:22.157407    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:18:26.332201 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:26.332172    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:26.332648 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:26.332252    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:43.731141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.731103    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:43.760412 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.760374    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:43.760619 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.760593    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" podUID="f39bb01f-910f-4426-820a-67df6cacbc1e" containerName="payload-processing" containerID="cri-o://7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838" gracePeriod=30
Jul 29 21:18:43.767838 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.767258    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:43.767838 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.767466    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" podUID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" containerName="payload-pre-processing" containerID="cri-o://a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b" gracePeriod=30
Jul 29 21:18:43.797769 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.797716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:43.907590 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.907546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:43.963501 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:43.963398    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:44.034038 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.033001    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:44.038086 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.038062    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:44.063182 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.063131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:44.164979 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.164953    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume\") pod \"f39bb01f-910f-4426-820a-67df6cacbc1e\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") "
Jul 29 21:18:44.165133 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.164984    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh7xh\" (UniqueName: \"kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh\") pod \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") "
Jul 29 21:18:44.165133 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.165046    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume\") pod \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\" (UID: \"d6f8401f-5ec4-4faa-a4ab-fb89330e827e\") "
Jul 29 21:18:44.165133 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.165063    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tndf7\" (UniqueName: \"kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7\") pod \"f39bb01f-910f-4426-820a-67df6cacbc1e\" (UID: \"f39bb01f-910f-4426-820a-67df6cacbc1e\") "
Jul 29 21:18:44.165388 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.165366    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f39bb01f-910f-4426-820a-67df6cacbc1e" (UID: "f39bb01f-910f-4426-820a-67df6cacbc1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:44.165446 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.165431    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d6f8401f-5ec4-4faa-a4ab-fb89330e827e" (UID: "d6f8401f-5ec4-4faa-a4ab-fb89330e827e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:18:44.167203 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.167177    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh" (OuterVolumeSpecName: "kube-api-access-rh7xh") pod "d6f8401f-5ec4-4faa-a4ab-fb89330e827e" (UID: "d6f8401f-5ec4-4faa-a4ab-fb89330e827e"). InnerVolumeSpecName "kube-api-access-rh7xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:44.167278 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.167209    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7" (OuterVolumeSpecName: "kube-api-access-tndf7") pod "f39bb01f-910f-4426-820a-67df6cacbc1e" (UID: "f39bb01f-910f-4426-820a-67df6cacbc1e"). InnerVolumeSpecName "kube-api-access-tndf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:18:44.266016 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.265958    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:44.266016 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.265982    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tndf7\" (UniqueName: \"kubernetes.io/projected/f39bb01f-910f-4426-820a-67df6cacbc1e-kube-api-access-tndf7\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:44.266016 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.265992    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f39bb01f-910f-4426-820a-67df6cacbc1e-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:44.266016 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.266000    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh7xh\" (UniqueName: \"kubernetes.io/projected/d6f8401f-5ec4-4faa-a4ab-fb89330e827e-kube-api-access-rh7xh\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:18:44.317790 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.317760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:44.457056 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.457026    2569 generic.go:356] "Generic (PLEG): container finished" podID="f39bb01f-910f-4426-820a-67df6cacbc1e" containerID="7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838" exitCode=0
Jul 29 21:18:44.457197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.457091    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"
Jul 29 21:18:44.457197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.457119    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" event={"ID":"f39bb01f-910f-4426-820a-67df6cacbc1e","Type":"ContainerDied","Data":"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"}
Jul 29 21:18:44.457311 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.457210    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc" event={"ID":"f39bb01f-910f-4426-820a-67df6cacbc1e","Type":"ContainerDied","Data":"0c7f63014a94f2143df7c6373d21e690749ae2169d6b0cd5218a71932d1227fd"}
Jul 29 21:18:44.457311 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.457243    2569 scope.go:117] "RemoveContainer" containerID="7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"
Jul 29 21:18:44.458422 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.458398    2569 generic.go:356] "Generic (PLEG): container finished" podID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" containerID="a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b" exitCode=0
Jul 29 21:18:44.458546 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.458445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" event={"ID":"d6f8401f-5ec4-4faa-a4ab-fb89330e827e","Type":"ContainerDied","Data":"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"}
Jul 29 21:18:44.458546 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.458454    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"
Jul 29 21:18:44.458546 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.458476    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp" event={"ID":"d6f8401f-5ec4-4faa-a4ab-fb89330e827e","Type":"ContainerDied","Data":"a0c359dd8d3a0adde625116077970fec97b0f2f33a4485d12641ffe15b23db3b"}
Jul 29 21:18:44.465827 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.465805    2569 scope.go:117] "RemoveContainer" containerID="7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"
Jul 29 21:18:44.466093 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:18:44.466070    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838\": container with ID starting with 7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838 not found: ID does not exist" containerID="7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"
Jul 29 21:18:44.466164 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.466103    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838"} err="failed to get container status \"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838\": rpc error: code = NotFound desc = could not find container \"7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838\": container with ID starting with 7b8edfc5b4b5ac705fce58a303d8d3eb0e722836832ad3f5bede8180a7208838 not found: ID does not exist"
Jul 29 21:18:44.466164 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.466126    2569 scope.go:117] "RemoveContainer" containerID="a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"
Jul 29 21:18:44.474963 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.474948    2569 scope.go:117] "RemoveContainer" containerID="a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"
Jul 29 21:18:44.475175 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:18:44.475157    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b\": container with ID starting with a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b not found: ID does not exist" containerID="a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"
Jul 29 21:18:44.475217 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.475181    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b"} err="failed to get container status \"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b\": rpc error: code = NotFound desc = could not find container \"a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b\": container with ID starting with a2f1c0b667d10ce239675c3f6b8014d8d018fdca8a869c5e774a888337ef065b not found: ID does not exist"
Jul 29 21:18:44.482252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.482231    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:44.486006 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.485987    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2d0e6b56-67d8bb9c9d-49rrc"]
Jul 29 21:18:44.495562 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.495544    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:44.499197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.499178    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2d0e6b56-7787455f68-vlqzp"]
Jul 29 21:18:44.810868 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.810838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:44.922070 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:44.922040    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:45.051347 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:45.051277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:45.900148 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:45.900109    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" path="/var/lib/kubelet/pods/d6f8401f-5ec4-4faa-a4ab-fb89330e827e/volumes"
Jul 29 21:18:45.900556 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:45.900481    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39bb01f-910f-4426-820a-67df6cacbc1e" path="/var/lib/kubelet/pods/f39bb01f-910f-4426-820a-67df6cacbc1e/volumes"
Jul 29 21:18:47.753138 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:18:47.753107    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:19:07.053000 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:07.052958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:08.858172 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:08.858132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:19:13.457928 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:13.457888    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:13.561520 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:13.561489    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:13.658833 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:13.658802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:13.758657 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:13.758631    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:14.209838 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:14.209756    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:40.147317 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:40.147284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:47.073663 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:47.073619    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:47.174731 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:19:47.174701    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:02.544845 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:02.544796    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.358780 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:08.358728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.459370 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:08.459341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:08.558914 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:08.558881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:19.081315 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:19.081277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:30.110902 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:30.110855    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:36.209173 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:36.209139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:36.310079 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:36.310043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:36.450236 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:36.450196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:36.747781 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:36.747729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:36.991534 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:36.991495    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:39.428405 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:39.428372    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:41.209604 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.209560    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:41.230982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.230950    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:20:41.231361 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231347    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39bb01f-910f-4426-820a-67df6cacbc1e" containerName="payload-processing"
Jul 29 21:20:41.231418 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231363    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39bb01f-910f-4426-820a-67df6cacbc1e" containerName="payload-processing"
Jul 29 21:20:41.231418 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231373    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" containerName="payload-pre-processing"
Jul 29 21:20:41.231418 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231379    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" containerName="payload-pre-processing"
Jul 29 21:20:41.231512 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231448    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39bb01f-910f-4426-820a-67df6cacbc1e" containerName="payload-processing"
Jul 29 21:20:41.231512 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.231458    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6f8401f-5ec4-4faa-a4ab-fb89330e827e" containerName="payload-pre-processing"
Jul 29 21:20:41.234296 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.234280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.236948 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.236926    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f7c67e2c-dockercfg-pczsv\""
Jul 29 21:20:41.237068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.236934    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f7c67e2c\""
Jul 29 21:20:41.243532 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.243504    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:20:41.263204 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.263166    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:20:41.266375 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.266353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.272027 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.272002    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8krc\" (UniqueName: \"kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.272158 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.272037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.277670 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.277642    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:20:41.373027 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.372989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8krc\" (UniqueName: \"kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.373197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.373048    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.373197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.373095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgs65\" (UniqueName: \"kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.373197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.373160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.373794 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.373777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.380604 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.380574    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8krc\" (UniqueName: \"kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc\") pod \"payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.474421 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.474392    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgs65\" (UniqueName: \"kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.474586 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.474461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.475255 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.475235    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.482802 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.482780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgs65\" (UniqueName: \"kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65\") pod \"payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") " pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.546385 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.546353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.578888 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.578837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:41.696692 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.696665    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:20:41.698980 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:20:41.698948    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c896ce_75d5_4e8c_bd51_deec53fab8c3.slice/crio-87e029e61a02813501742d77107f76058d48ebbdbacb470c677c4302359284de WatchSource:0}: Error finding container 87e029e61a02813501742d77107f76058d48ebbdbacb470c677c4302359284de: Status 404 returned error can't find the container with id 87e029e61a02813501742d77107f76058d48ebbdbacb470c677c4302359284de
Jul 29 21:20:41.732296 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.732253    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:41.733186 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.733106    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:20:41.735247 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:20:41.735227    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fa73f6_63e7_4966_87b2_01ced9ce217c.slice/crio-4d029286d928f836b60f586f7b7767cb1bfc0e5ce1821addebe50bdde2451f3c WatchSource:0}: Error finding container 4d029286d928f836b60f586f7b7767cb1bfc0e5ce1821addebe50bdde2451f3c: Status 404 returned error can't find the container with id 4d029286d928f836b60f586f7b7767cb1bfc0e5ce1821addebe50bdde2451f3c
Jul 29 21:20:41.987523 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.987433    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" event={"ID":"00c896ce-75d5-4e8c-bd51-deec53fab8c3","Type":"ContainerStarted","Data":"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"}
Jul 29 21:20:41.987523 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.987490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" event={"ID":"00c896ce-75d5-4e8c-bd51-deec53fab8c3","Type":"ContainerStarted","Data":"87e029e61a02813501742d77107f76058d48ebbdbacb470c677c4302359284de"}
Jul 29 21:20:41.987523 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.987519    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:41.989489 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.989459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" event={"ID":"b0fa73f6-63e7-4966-87b2-01ced9ce217c","Type":"ContainerStarted","Data":"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"}
Jul 29 21:20:41.989623 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.989505    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" event={"ID":"b0fa73f6-63e7-4966-87b2-01ced9ce217c","Type":"ContainerStarted","Data":"4d029286d928f836b60f586f7b7767cb1bfc0e5ce1821addebe50bdde2451f3c"}
Jul 29 21:20:41.989623 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:41.989537    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:20:42.004676 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:42.004617    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" podStartSLOduration=1.004596699 podStartE2EDuration="1.004596699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:20:42.003261167 +0000 UTC m=+2054.601237538" watchObservedRunningTime="2026-07-29 21:20:42.004596699 +0000 UTC m=+2054.602573060"
Jul 29 21:20:42.018909 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:42.018863    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" podStartSLOduration=1.018849816 podStartE2EDuration="1.018849816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:20:42.01858874 +0000 UTC m=+2054.616565097" watchObservedRunningTime="2026-07-29 21:20:42.018849816 +0000 UTC m=+2054.616826173"
Jul 29 21:20:45.650762 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:45.650701    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:20:52.996636 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:52.996553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:20:52.997126 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:20:52.996640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:21:04.448052 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:04.447996    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:15.922057 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:15.922015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:16.259225 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:16.259184    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:16.395720 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:16.395684    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:19.588576 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:19.588542    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:19.687416 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:19.687382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:20.679106 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.678811    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:21:20.683448 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.683423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.686107 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.686079    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:21:20.686265 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.686082    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-58463a07-serving-cert\""
Jul 29 21:21:20.686265 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.686137    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:21:20.687289 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.687266    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:21:20.691758 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.691712    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:21:20.724456 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.724426    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:20.790649 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.790625    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.790770 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.790652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2csb\" (UniqueName: \"kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.790770 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.790724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.891295 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.891270    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.891393 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.891318    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.891393 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.891339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2csb\" (UniqueName: \"kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.891469 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:21:20.891406    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-serving-cert: secret "maas-api-e2e-mt-58463a07-serving-cert" not found
Jul 29 21:21:20.891510 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:21:20.891477    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls podName:cc5404cb-458e-49c7-a750-7968b7d7013d nodeName:}" failed. No retries permitted until 2026-07-29 21:21:21.391459226 +0000 UTC m=+2093.989435561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls") pod "maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" (UID: "cc5404cb-458e-49c7-a750-7968b7d7013d") : secret "maas-api-e2e-mt-58463a07-serving-cert" not found
Jul 29 21:21:20.891969 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.891952    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:20.901455 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:20.901434    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2csb\" (UniqueName: \"kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:21.078354 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.078330    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:21.221463 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.221429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:21.313717 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.313675    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:21.320106 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.320080    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:21:21.323684 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.323663    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.326304 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.326280    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-58463a07-dockercfg-qwwfp\""
Jul 29 21:21:21.326431 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.326329    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-58463a07\""
Jul 29 21:21:21.334693 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.334664    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:21:21.353576 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.353549    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:21:21.357318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.357302    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.364246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.364222    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:21:21.394716 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.394690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:21.394839 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.394799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf98h\" (UniqueName: \"kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.394904 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.394885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.397117 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.397095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") pod \"maas-api-e2e-mt-58463a07-57b4f949b-zx9cq\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:21.496124 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.496098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87z4t\" (UniqueName: \"kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.496250 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.496140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.496328 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.496251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf98h\" (UniqueName: \"kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.496328 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.496314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.496940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.496923    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.503852 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.503835    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf98h\" (UniqueName: \"kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h\") pod \"payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.596854 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.596802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.596854 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.596838    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87z4t\" (UniqueName: \"kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.597427 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.597408    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.598778 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.598759    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:21.604759 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.604725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87z4t\" (UniqueName: \"kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t\") pod \"payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") " pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.635747 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.635702    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:21.669343 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.669095    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:21.742436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.742371    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:21:21.748170 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:21:21.748135    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc5404cb_458e_49c7_a750_7968b7d7013d.slice/crio-8c724307a916358629aea28850f4225e80bdc7f6913a33fc4c5cbeba19254623 WatchSource:0}: Error finding container 8c724307a916358629aea28850f4225e80bdc7f6913a33fc4c5cbeba19254623: Status 404 returned error can't find the container with id 8c724307a916358629aea28850f4225e80bdc7f6913a33fc4c5cbeba19254623
Jul 29 21:21:21.798047 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.798019    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:21:21.800020 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:21:21.799986    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae856a3_ea3a_437d_b4cd_80563baf5260.slice/crio-030c3a209edd286e838118e11bdf5c3d32c693b373e04afec90b100499f763c6 WatchSource:0}: Error finding container 030c3a209edd286e838118e11bdf5c3d32c693b373e04afec90b100499f763c6: Status 404 returned error can't find the container with id 030c3a209edd286e838118e11bdf5c3d32c693b373e04afec90b100499f763c6
Jul 29 21:21:21.823379 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:21.823341    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:21:21.832609 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:21:21.832582    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c126df1_4e91_46da_8899_711f5a4c879d.slice/crio-bc8a16e1b7910e84fb4975df502c7d909f81d0c44df5eebc87ffef37e634ab1a WatchSource:0}: Error finding container bc8a16e1b7910e84fb4975df502c7d909f81d0c44df5eebc87ffef37e634ab1a: Status 404 returned error can't find the container with id bc8a16e1b7910e84fb4975df502c7d909f81d0c44df5eebc87ffef37e634ab1a
Jul 29 21:21:22.169165 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.169086    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" event={"ID":"0ae856a3-ea3a-437d-b4cd-80563baf5260","Type":"ContainerStarted","Data":"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"}
Jul 29 21:21:22.169165 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.169150    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:22.169366 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.169180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" event={"ID":"0ae856a3-ea3a-437d-b4cd-80563baf5260","Type":"ContainerStarted","Data":"030c3a209edd286e838118e11bdf5c3d32c693b373e04afec90b100499f763c6"}
Jul 29 21:21:22.170395 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.170371    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" event={"ID":"cc5404cb-458e-49c7-a750-7968b7d7013d","Type":"ContainerStarted","Data":"8c724307a916358629aea28850f4225e80bdc7f6913a33fc4c5cbeba19254623"}
Jul 29 21:21:22.171704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.171683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" event={"ID":"4c126df1-4e91-46da-8899-711f5a4c879d","Type":"ContainerStarted","Data":"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"}
Jul 29 21:21:22.171825 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.171718    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" event={"ID":"4c126df1-4e91-46da-8899-711f5a4c879d","Type":"ContainerStarted","Data":"bc8a16e1b7910e84fb4975df502c7d909f81d0c44df5eebc87ffef37e634ab1a"}
Jul 29 21:21:22.171825 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.171786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:22.185331 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.185294    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" podStartSLOduration=1.185283821 podStartE2EDuration="1.185283821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:21:22.18353006 +0000 UTC m=+2094.781506418" watchObservedRunningTime="2026-07-29 21:21:22.185283821 +0000 UTC m=+2094.783260178"
Jul 29 21:21:22.201700 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:22.201657    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" podStartSLOduration=1.201645231 podStartE2EDuration="1.201645231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:21:22.200695668 +0000 UTC m=+2094.798672050" watchObservedRunningTime="2026-07-29 21:21:22.201645231 +0000 UTC m=+2094.799621588"
Jul 29 21:21:24.182039 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:24.182004    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" event={"ID":"cc5404cb-458e-49c7-a750-7968b7d7013d","Type":"ContainerStarted","Data":"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"}
Jul 29 21:21:24.182465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:24.182119    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:24.200941 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:24.200900    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" podStartSLOduration=1.940697565 podStartE2EDuration="4.200885189s" totalImagesPullingTime="2.260187624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:21:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:21:24.198834197 +0000 UTC m=+2096.796810566" watchObservedRunningTime="2026-07-29 21:21:24.200885189 +0000 UTC m=+2096.798861547"
Jul 29 21:21:27.233135 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:27.233090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:27.661783 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:27.661673    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:21:30.193222 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:30.193192    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:21:33.178273 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:33.178240    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:21:33.178664 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:33.178314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:21:44.650344 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:21:44.650300    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:06.555199 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:24:06.555116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:10.536368 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:27:10.536258    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:12.380913 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:27:12.380879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.461315 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:27:25.461275    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.519922 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:27:25.519889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:31.002145 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:27:31.002110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:09.026603 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:28:09.026554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:31.181095 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:28:31.180955    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:37.252560 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:28:37.252526    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:01.471020 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:01.470894    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:01.674824 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:01.674798    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:04.858020 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:04.857988    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:30:17.264551 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.264515    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:30:17.265007 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.264700    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" podUID="cc5404cb-458e-49c7-a750-7968b7d7013d" containerName="maas-api" containerID="cri-o://b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a" gracePeriod=30
Jul 29 21:30:17.299800 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.299776    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:17.330239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.330211    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:30:17.330465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.330439    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" podUID="4c126df1-4e91-46da-8899-711f5a4c879d" containerName="payload-processing" containerID="cri-o://28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249" gracePeriod=30
Jul 29 21:30:17.341500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.341418    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:30:17.341832 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.341651    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" podUID="0ae856a3-ea3a-437d-b4cd-80563baf5260" containerName="payload-pre-processing" containerID="cri-o://b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d" gracePeriod=30
Jul 29 21:30:17.378080 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.378055    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:17.482864 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.482728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:17.588901 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.588867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:17.615352 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.615331    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:30:17.622244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.622225    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:30:17.632668 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.632649    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:30:17.674386 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.674352    2569 generic.go:356] "Generic (PLEG): container finished" podID="0ae856a3-ea3a-437d-b4cd-80563baf5260" containerID="b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d" exitCode=0
Jul 29 21:30:17.674577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.674392    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"
Jul 29 21:30:17.674577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.674442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" event={"ID":"0ae856a3-ea3a-437d-b4cd-80563baf5260","Type":"ContainerDied","Data":"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"}
Jul 29 21:30:17.674577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.674501    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx" event={"ID":"0ae856a3-ea3a-437d-b4cd-80563baf5260","Type":"ContainerDied","Data":"030c3a209edd286e838118e11bdf5c3d32c693b373e04afec90b100499f763c6"}
Jul 29 21:30:17.674577 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.674530    2569 scope.go:117] "RemoveContainer" containerID="b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"
Jul 29 21:30:17.676374 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.676346    2569 generic.go:356] "Generic (PLEG): container finished" podID="cc5404cb-458e-49c7-a750-7968b7d7013d" containerID="b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a" exitCode=0
Jul 29 21:30:17.676468 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.676391    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"
Jul 29 21:30:17.676539 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.676467    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" event={"ID":"cc5404cb-458e-49c7-a750-7968b7d7013d","Type":"ContainerDied","Data":"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"}
Jul 29 21:30:17.677189 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.677139    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq" event={"ID":"cc5404cb-458e-49c7-a750-7968b7d7013d","Type":"ContainerDied","Data":"8c724307a916358629aea28850f4225e80bdc7f6913a33fc4c5cbeba19254623"}
Jul 29 21:30:17.679470 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.679447    2569 generic.go:356] "Generic (PLEG): container finished" podID="4c126df1-4e91-46da-8899-711f5a4c879d" containerID="28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249" exitCode=0
Jul 29 21:30:17.679588 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.679493    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"
Jul 29 21:30:17.679588 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.679512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" event={"ID":"4c126df1-4e91-46da-8899-711f5a4c879d","Type":"ContainerDied","Data":"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"}
Jul 29 21:30:17.679588 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.679549    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck" event={"ID":"4c126df1-4e91-46da-8899-711f5a4c879d","Type":"ContainerDied","Data":"bc8a16e1b7910e84fb4975df502c7d909f81d0c44df5eebc87ffef37e634ab1a"}
Jul 29 21:30:17.685036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.685019    2569 scope.go:117] "RemoveContainer" containerID="b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"
Jul 29 21:30:17.685337 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:17.685314    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d\": container with ID starting with b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d not found: ID does not exist" containerID="b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"
Jul 29 21:30:17.685411 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.685343    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d"} err="failed to get container status \"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d\": rpc error: code = NotFound desc = could not find container \"b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d\": container with ID starting with b70d1fdb4ff269d62a898570202676d42b0c6b081d15cc81a3529cfca8e36a5d not found: ID does not exist"
Jul 29 21:30:17.685411 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.685358    2569 scope.go:117] "RemoveContainer" containerID="b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"
Jul 29 21:30:17.699031 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.699011    2569 scope.go:117] "RemoveContainer" containerID="b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"
Jul 29 21:30:17.699310 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:17.699287    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a\": container with ID starting with b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a not found: ID does not exist" containerID="b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"
Jul 29 21:30:17.699362 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.699317    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a"} err="failed to get container status \"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a\": rpc error: code = NotFound desc = could not find container \"b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a\": container with ID starting with b013c9ee8657285de8a8a0f4ef2f8543e0c2e5692a3d7e5e00029b3c4187d37a not found: ID does not exist"
Jul 29 21:30:17.699362 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.699335    2569 scope.go:117] "RemoveContainer" containerID="28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"
Jul 29 21:30:17.705437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705412    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") pod \"cc5404cb-458e-49c7-a750-7968b7d7013d\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") "
Jul 29 21:30:17.705538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705451    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2csb\" (UniqueName: \"kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb\") pod \"cc5404cb-458e-49c7-a750-7968b7d7013d\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") "
Jul 29 21:30:17.705538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705472    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87z4t\" (UniqueName: \"kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t\") pod \"4c126df1-4e91-46da-8899-711f5a4c879d\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") "
Jul 29 21:30:17.705538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705492    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca\") pod \"cc5404cb-458e-49c7-a750-7968b7d7013d\" (UID: \"cc5404cb-458e-49c7-a750-7968b7d7013d\") "
Jul 29 21:30:17.705538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705522    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume\") pod \"0ae856a3-ea3a-437d-b4cd-80563baf5260\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") "
Jul 29 21:30:17.705776 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705583    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume\") pod \"4c126df1-4e91-46da-8899-711f5a4c879d\" (UID: \"4c126df1-4e91-46da-8899-711f5a4c879d\") "
Jul 29 21:30:17.705776 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705612    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf98h\" (UniqueName: \"kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h\") pod \"0ae856a3-ea3a-437d-b4cd-80563baf5260\" (UID: \"0ae856a3-ea3a-437d-b4cd-80563baf5260\") "
Jul 29 21:30:17.705950 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.705925    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc5404cb-458e-49c7-a750-7968b7d7013d" (UID: "cc5404cb-458e-49c7-a750-7968b7d7013d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:17.706355 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.706316    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ae856a3-ea3a-437d-b4cd-80563baf5260" (UID: "0ae856a3-ea3a-437d-b4cd-80563baf5260"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:17.706484 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.706458    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c126df1-4e91-46da-8899-711f5a4c879d" (UID: "4c126df1-4e91-46da-8899-711f5a4c879d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:17.707678 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.707658    2569 scope.go:117] "RemoveContainer" containerID="28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"
Jul 29 21:30:17.707771 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.707673    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t" (OuterVolumeSpecName: "kube-api-access-87z4t") pod "4c126df1-4e91-46da-8899-711f5a4c879d" (UID: "4c126df1-4e91-46da-8899-711f5a4c879d"). InnerVolumeSpecName "kube-api-access-87z4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:17.707888 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.707859    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb" (OuterVolumeSpecName: "kube-api-access-m2csb") pod "cc5404cb-458e-49c7-a750-7968b7d7013d" (UID: "cc5404cb-458e-49c7-a750-7968b7d7013d"). InnerVolumeSpecName "kube-api-access-m2csb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:17.707996 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:17.707969    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249\": container with ID starting with 28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249 not found: ID does not exist" containerID="28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"
Jul 29 21:30:17.708049 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.707992    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249"} err="failed to get container status \"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249\": rpc error: code = NotFound desc = could not find container \"28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249\": container with ID starting with 28039365b9bd9a1e3fbaed7bb48979006301bb4cec1e8a64609fe00051ea8249 not found: ID does not exist"
Jul 29 21:30:17.708049 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.707980    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h" (OuterVolumeSpecName: "kube-api-access-vf98h") pod "0ae856a3-ea3a-437d-b4cd-80563baf5260" (UID: "0ae856a3-ea3a-437d-b4cd-80563baf5260"). InnerVolumeSpecName "kube-api-access-vf98h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:17.708049 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.708018    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc5404cb-458e-49c7-a750-7968b7d7013d" (UID: "cc5404cb-458e-49c7-a750-7968b7d7013d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806264    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc5404cb-458e-49c7-a750-7968b7d7013d-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806290    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2csb\" (UniqueName: \"kubernetes.io/projected/cc5404cb-458e-49c7-a750-7968b7d7013d-kube-api-access-m2csb\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806301    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87z4t\" (UniqueName: \"kubernetes.io/projected/4c126df1-4e91-46da-8899-711f5a4c879d-kube-api-access-87z4t\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806311    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc5404cb-458e-49c7-a750-7968b7d7013d-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806320    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ae856a3-ea3a-437d-b4cd-80563baf5260-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806328    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c126df1-4e91-46da-8899-711f5a4c879d-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.806337 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.806339    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf98h\" (UniqueName: \"kubernetes.io/projected/0ae856a3-ea3a-437d-b4cd-80563baf5260-kube-api-access-vf98h\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:17.927047 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.927022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:17.994283 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.994258    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:30:17.999546 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:17.999523    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58463a07-57b4f949b-zx9cq"]
Jul 29 21:30:18.018868 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:18.018847    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:30:18.021176 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:18.021154    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58463a07-5d7564f5b4-4dzfx"]
Jul 29 21:30:18.035254 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:18.035230    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:30:18.040604 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:18.040585    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58463a07-5ffb649c6b-bngck"]
Jul 29 21:30:19.898333 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:19.898296    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ae856a3-ea3a-437d-b4cd-80563baf5260" path="/var/lib/kubelet/pods/0ae856a3-ea3a-437d-b4cd-80563baf5260/volumes"
Jul 29 21:30:19.898851 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:19.898668    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c126df1-4e91-46da-8899-711f5a4c879d" path="/var/lib/kubelet/pods/4c126df1-4e91-46da-8899-711f5a4c879d/volumes"
Jul 29 21:30:19.899111 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:19.899090    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc5404cb-458e-49c7-a750-7968b7d7013d" path="/var/lib/kubelet/pods/cc5404cb-458e-49c7-a750-7968b7d7013d/volumes"
Jul 29 21:30:19.961499 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:19.961467    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:30:23.833327 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:23.833289    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:23.957713 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:23.957673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:24.095634 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:24.095560    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:24.331104 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:24.331071    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:24.684037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:24.684009    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:24.991955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:24.991920    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:25.374177 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:25.374101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:28.088913 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:28.088881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:28.233888 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:28.233858    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:29.185001 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.184966    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185334    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ae856a3-ea3a-437d-b4cd-80563baf5260" containerName="payload-pre-processing"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185345    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ae856a3-ea3a-437d-b4cd-80563baf5260" containerName="payload-pre-processing"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185353    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c126df1-4e91-46da-8899-711f5a4c879d" containerName="payload-processing"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185358    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c126df1-4e91-46da-8899-711f5a4c879d" containerName="payload-processing"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185370    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc5404cb-458e-49c7-a750-7968b7d7013d" containerName="maas-api"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185375    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc5404cb-458e-49c7-a750-7968b7d7013d" containerName="maas-api"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185425    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc5404cb-458e-49c7-a750-7968b7d7013d" containerName="maas-api"
Jul 29 21:30:29.185432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185434    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c126df1-4e91-46da-8899-711f5a4c879d" containerName="payload-processing"
Jul 29 21:30:29.185876 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.185442    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ae856a3-ea3a-437d-b4cd-80563baf5260" containerName="payload-pre-processing"
Jul 29 21:30:29.190375 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.190353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.195543 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.195524    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a9af2a72-serving-cert\""
Jul 29 21:30:29.195803 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.195545    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:30:29.195926 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.195677    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:30:29.196658 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.196640    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:30:29.203496 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.203475    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:29.247593 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.247565    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:29.288813 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.288787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.288905 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.288828    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vgr8\" (UniqueName: \"kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.288963 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.288913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.353322 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.353298    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:29.389392 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.389371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.389500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.389417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.389500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.389441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vgr8\" (UniqueName: \"kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.390084 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.390060    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.392090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.392067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.400541 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.400517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vgr8\" (UniqueName: \"kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8\") pod \"maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.501803 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.501780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:29.629252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.629230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:29.631476 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:30:29.631448    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231806fe_906a_427b_b441_5e31e2c48803.slice/crio-f689daa6acb2c9f9c81429da07a0469e1f88be527a2dff7afa49bc724d9253a5 WatchSource:0}: Error finding container f689daa6acb2c9f9c81429da07a0469e1f88be527a2dff7afa49bc724d9253a5: Status 404 returned error can't find the container with id f689daa6acb2c9f9c81429da07a0469e1f88be527a2dff7afa49bc724d9253a5
Jul 29 21:30:29.633294 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.633271    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:30:29.742853 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.742823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" event={"ID":"231806fe-906a-427b-b441-5e31e2c48803","Type":"ContainerStarted","Data":"f689daa6acb2c9f9c81429da07a0469e1f88be527a2dff7afa49bc724d9253a5"}
Jul 29 21:30:29.826363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.826296    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:29.927671 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:29.927646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:30.008089 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.008059    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:30.012348 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.012332    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.014936 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.014917    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a9af2a72-dockercfg-9vkrt\""
Jul 29 21:30:30.015044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.014953    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a9af2a72\""
Jul 29 21:30:30.019067 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.019043    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:30.056730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.056710    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:30.059959 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.059942    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.074090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.074069    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:30.095246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.095189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.095246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.095231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wwb8\" (UniqueName: \"kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.195581 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.195555    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.195989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.195633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.195989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.195688    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr6x5\" (UniqueName: \"kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.195989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.195723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wwb8\" (UniqueName: \"kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.196432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.196414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.204692 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.204669    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wwb8\" (UniqueName: \"kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8\") pod \"payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.297059 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.297029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr6x5\" (UniqueName: \"kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.297178 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.297089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.297731 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.297713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.306786 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.306763    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr6x5\" (UniqueName: \"kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5\") pod \"payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") " pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.324594 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.324573    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.368909 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.368841    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.460043 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.460016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:30.463076 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:30:30.463046    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111c16ae_1045_471f_aa7a_9a02077f4d49.slice/crio-2e41e57c85525406c8f96c1acc699b623db663a52867d4395ea0a9c3bf7dfcac WatchSource:0}: Error finding container 2e41e57c85525406c8f96c1acc699b623db663a52867d4395ea0a9c3bf7dfcac: Status 404 returned error can't find the container with id 2e41e57c85525406c8f96c1acc699b623db663a52867d4395ea0a9c3bf7dfcac
Jul 29 21:30:30.514805 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.514775    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:30.518312 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:30:30.518289    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb3e4a6_a3b0_4bae_bb9b_170388f35208.slice/crio-920700c4fe3d587776e17376ae59adc8af9b6c474e60c57d3f0dfb5b48216651 WatchSource:0}: Error finding container 920700c4fe3d587776e17376ae59adc8af9b6c474e60c57d3f0dfb5b48216651: Status 404 returned error can't find the container with id 920700c4fe3d587776e17376ae59adc8af9b6c474e60c57d3f0dfb5b48216651
Jul 29 21:30:30.749179 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.749148    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" event={"ID":"231806fe-906a-427b-b441-5e31e2c48803","Type":"ContainerStarted","Data":"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"}
Jul 29 21:30:30.749349 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.749231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:30.750659 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.750639    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" event={"ID":"2bb3e4a6-a3b0-4bae-bb9b-170388f35208","Type":"ContainerStarted","Data":"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"}
Jul 29 21:30:30.750791 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.750680    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:30.750791 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.750705    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" event={"ID":"2bb3e4a6-a3b0-4bae-bb9b-170388f35208","Type":"ContainerStarted","Data":"920700c4fe3d587776e17376ae59adc8af9b6c474e60c57d3f0dfb5b48216651"}
Jul 29 21:30:30.752113 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.752087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" event={"ID":"111c16ae-1045-471f-aa7a-9a02077f4d49","Type":"ContainerStarted","Data":"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"}
Jul 29 21:30:30.752198 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.752139    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:30.752198 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.752162    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" event={"ID":"111c16ae-1045-471f-aa7a-9a02077f4d49","Type":"ContainerStarted","Data":"2e41e57c85525406c8f96c1acc699b623db663a52867d4395ea0a9c3bf7dfcac"}
Jul 29 21:30:30.769308 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.769268    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" podStartSLOduration=1.199950389 podStartE2EDuration="1.76925489s" totalImagesPullingTime="569.304501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:30.767680066 +0000 UTC m=+2643.365656425" watchObservedRunningTime="2026-07-29 21:30:30.76925489 +0000 UTC m=+2643.367231247"
Jul 29 21:30:30.783619 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.783585    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" podStartSLOduration=1.783574569 podStartE2EDuration="1.783574569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:30.783465644 +0000 UTC m=+2643.381442004" watchObservedRunningTime="2026-07-29 21:30:30.783574569 +0000 UTC m=+2643.381550936"
Jul 29 21:30:30.805023 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:30.804984    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" podStartSLOduration=0.804971671 podStartE2EDuration="804.971671ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:30:30.803495212 +0000 UTC m=+2643.401471570" watchObservedRunningTime="2026-07-29 21:30:30.804971671 +0000 UTC m=+2643.402948027"
Jul 29 21:30:32.968219 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:32.968186    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:30:36.761937 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:36.761905    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:37.714376 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:37.714343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:38.477908 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:38.477877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:38.574656 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:38.574628    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:38.932987 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:38.932917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:41.759047 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:41.759019    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:41.759393 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:41.759085    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:45.227882 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:45.227846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:45.327464 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:45.327437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:50.973116 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:50.973088    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:50.973559 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:50.973284    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" podUID="231806fe-906a-427b-b441-5e31e2c48803" containerName="maas-api" containerID="cri-o://0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd" gracePeriod=30
Jul 29 21:30:51.010151 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.010128    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.040329 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.040305    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:51.040536 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.040514    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" podUID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" containerName="payload-processing" containerID="cri-o://01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d" gracePeriod=30
Jul 29 21:30:51.052960 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.052935    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:51.053120 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.053097    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" podUID="111c16ae-1045-471f-aa7a-9a02077f4d49" containerName="payload-pre-processing" containerID="cri-o://e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d" gracePeriod=30
Jul 29 21:30:51.107598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.107571    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.214912 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.214761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.311846 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.311824    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:51.335150 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.335124    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:51.343965 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.343944    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:51.364110 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.364084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.449833 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.449806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume\") pod \"111c16ae-1045-471f-aa7a-9a02077f4d49\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") "
Jul 29 21:30:51.449940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.449918    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume\") pod \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") "
Jul 29 21:30:51.449982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.449948    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rr6x5\" (UniqueName: \"kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5\") pod \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\" (UID: \"2bb3e4a6-a3b0-4bae-bb9b-170388f35208\") "
Jul 29 21:30:51.449982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.449967    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vgr8\" (UniqueName: \"kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8\") pod \"231806fe-906a-427b-b441-5e31e2c48803\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") "
Jul 29 21:30:51.450065 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.449993    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wwb8\" (UniqueName: \"kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8\") pod \"111c16ae-1045-471f-aa7a-9a02077f4d49\" (UID: \"111c16ae-1045-471f-aa7a-9a02077f4d49\") "
Jul 29 21:30:51.450065 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450020    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca\") pod \"231806fe-906a-427b-b441-5e31e2c48803\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") "
Jul 29 21:30:51.450266 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450238    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls\") pod \"231806fe-906a-427b-b441-5e31e2c48803\" (UID: \"231806fe-906a-427b-b441-5e31e2c48803\") "
Jul 29 21:30:51.450349 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450279    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "111c16ae-1045-471f-aa7a-9a02077f4d49" (UID: "111c16ae-1045-471f-aa7a-9a02077f4d49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:51.450349 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450309    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bb3e4a6-a3b0-4bae-bb9b-170388f35208" (UID: "2bb3e4a6-a3b0-4bae-bb9b-170388f35208"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:51.450457 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450375    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "231806fe-906a-427b-b441-5e31e2c48803" (UID: "231806fe-906a-427b-b441-5e31e2c48803"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:30:51.450597 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450578    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/111c16ae-1045-471f-aa7a-9a02077f4d49-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.450648 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450604    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.450648 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.450618    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/231806fe-906a-427b-b441-5e31e2c48803-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.452099 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.452080    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5" (OuterVolumeSpecName: "kube-api-access-rr6x5") pod "2bb3e4a6-a3b0-4bae-bb9b-170388f35208" (UID: "2bb3e4a6-a3b0-4bae-bb9b-170388f35208"). InnerVolumeSpecName "kube-api-access-rr6x5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:51.452171 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.452104    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8" (OuterVolumeSpecName: "kube-api-access-6vgr8") pod "231806fe-906a-427b-b441-5e31e2c48803" (UID: "231806fe-906a-427b-b441-5e31e2c48803"). InnerVolumeSpecName "kube-api-access-6vgr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:51.452434 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.452416    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "231806fe-906a-427b-b441-5e31e2c48803" (UID: "231806fe-906a-427b-b441-5e31e2c48803"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:30:51.452471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.452425    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8" (OuterVolumeSpecName: "kube-api-access-9wwb8") pod "111c16ae-1045-471f-aa7a-9a02077f4d49" (UID: "111c16ae-1045-471f-aa7a-9a02077f4d49"). InnerVolumeSpecName "kube-api-access-9wwb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:30:51.551610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.551543    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/231806fe-906a-427b-b441-5e31e2c48803-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.551610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.551576    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rr6x5\" (UniqueName: \"kubernetes.io/projected/2bb3e4a6-a3b0-4bae-bb9b-170388f35208-kube-api-access-rr6x5\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.551610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.551590    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vgr8\" (UniqueName: \"kubernetes.io/projected/231806fe-906a-427b-b441-5e31e2c48803-kube-api-access-6vgr8\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.551610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.551602    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wwb8\" (UniqueName: \"kubernetes.io/projected/111c16ae-1045-471f-aa7a-9a02077f4d49-kube-api-access-9wwb8\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:30:51.695695 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.695656    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:51.850496 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.850443    2569 generic.go:356] "Generic (PLEG): container finished" podID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" containerID="01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d" exitCode=0
Jul 29 21:30:51.850496 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.850475    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"
Jul 29 21:30:51.850650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.850515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" event={"ID":"2bb3e4a6-a3b0-4bae-bb9b-170388f35208","Type":"ContainerDied","Data":"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"}
Jul 29 21:30:51.850650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.850585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n" event={"ID":"2bb3e4a6-a3b0-4bae-bb9b-170388f35208","Type":"ContainerDied","Data":"920700c4fe3d587776e17376ae59adc8af9b6c474e60c57d3f0dfb5b48216651"}
Jul 29 21:30:51.850650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.850610    2569 scope.go:117] "RemoveContainer" containerID="01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"
Jul 29 21:30:51.851788 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.851766    2569 generic.go:356] "Generic (PLEG): container finished" podID="111c16ae-1045-471f-aa7a-9a02077f4d49" containerID="e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d" exitCode=0
Jul 29 21:30:51.851872 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.851811    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"
Jul 29 21:30:51.851872 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.851849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" event={"ID":"111c16ae-1045-471f-aa7a-9a02077f4d49","Type":"ContainerDied","Data":"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"}
Jul 29 21:30:51.851988 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.851898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7" event={"ID":"111c16ae-1045-471f-aa7a-9a02077f4d49","Type":"ContainerDied","Data":"2e41e57c85525406c8f96c1acc699b623db663a52867d4395ea0a9c3bf7dfcac"}
Jul 29 21:30:51.853272 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.853248    2569 generic.go:356] "Generic (PLEG): container finished" podID="231806fe-906a-427b-b441-5e31e2c48803" containerID="0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd" exitCode=0
Jul 29 21:30:51.853365 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.853295    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"
Jul 29 21:30:51.853427 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.853305    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" event={"ID":"231806fe-906a-427b-b441-5e31e2c48803","Type":"ContainerDied","Data":"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"}
Jul 29 21:30:51.853427 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.853416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks" event={"ID":"231806fe-906a-427b-b441-5e31e2c48803","Type":"ContainerDied","Data":"f689daa6acb2c9f9c81429da07a0469e1f88be527a2dff7afa49bc724d9253a5"}
Jul 29 21:30:51.859554 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.859534    2569 scope.go:117] "RemoveContainer" containerID="01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"
Jul 29 21:30:51.859833 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:51.859810    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d\": container with ID starting with 01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d not found: ID does not exist" containerID="01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"
Jul 29 21:30:51.859897 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.859844    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d"} err="failed to get container status \"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d\": rpc error: code = NotFound desc = could not find container \"01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d\": container with ID starting with 01d5db5efb9ad87bf8aeafa49c858e7583b3d1fab4f9ee61434555cb8728991d not found: ID does not exist"
Jul 29 21:30:51.859897 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.859869    2569 scope.go:117] "RemoveContainer" containerID="e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"
Jul 29 21:30:51.868609 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.868595    2569 scope.go:117] "RemoveContainer" containerID="e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"
Jul 29 21:30:51.868878 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:51.868862    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d\": container with ID starting with e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d not found: ID does not exist" containerID="e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"
Jul 29 21:30:51.868943 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.868883    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d"} err="failed to get container status \"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d\": rpc error: code = NotFound desc = could not find container \"e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d\": container with ID starting with e0c7c7f73a7adbee0d066e0270b958b9b95f1e9f37e49453d319442204342e2d not found: ID does not exist"
Jul 29 21:30:51.868943 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.868900    2569 scope.go:117] "RemoveContainer" containerID="0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"
Jul 29 21:30:51.875694 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.875669    2569 scope.go:117] "RemoveContainer" containerID="0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"
Jul 29 21:30:51.876005 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:30:51.875985    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd\": container with ID starting with 0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd not found: ID does not exist" containerID="0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"
Jul 29 21:30:51.876091 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.876012    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd"} err="failed to get container status \"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd\": rpc error: code = NotFound desc = could not find container \"0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd\": container with ID starting with 0f0a429bee36b60c6de4c000c04f6a5cdb41e53e4621a0b7e8fc8569f19b66cd not found: ID does not exist"
Jul 29 21:30:51.878776 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.878755    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:51.882995 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.882978    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a9af2a72-68c9bd5f8c-4wdks"]
Jul 29 21:30:51.897663 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.897644    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="231806fe-906a-427b-b441-5e31e2c48803" path="/var/lib/kubelet/pods/231806fe-906a-427b-b441-5e31e2c48803/volumes"
Jul 29 21:30:51.897903 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.897890    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:51.901566 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.901549    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a9af2a72-6857c6486b-lbn8n"]
Jul 29 21:30:51.912976 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.912948    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:51.914678 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:51.914658    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a9af2a72-5d66d9bdfb-cc6z7"]
Jul 29 21:30:53.867624 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:53.867585    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:30:53.902002 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:53.901133    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="111c16ae-1045-471f-aa7a-9a02077f4d49" path="/var/lib/kubelet/pods/111c16ae-1045-471f-aa7a-9a02077f4d49/volumes"
Jul 29 21:30:53.902666 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:53.902643    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" path="/var/lib/kubelet/pods/2bb3e4a6-a3b0-4bae-bb9b-170388f35208/volumes"
Jul 29 21:30:57.411578 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:57.411545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:57.515948 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:57.515925    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:57.774843 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:57.774811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:57.951856 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:57.951829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:58.090686 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:58.090618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:58.190259 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:58.190226    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:58.313805 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:30:58.313782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:04.210748 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:04.210710    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:04.287033 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:04.287002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:04.381063 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:04.381036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:04.580486 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:04.580458    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:11.183915 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:11.183883    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:11.283301 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:11.283274    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:31:30.584245 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:31:30.584210    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:33:12.904647 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:12.904608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:14.291702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:14.291666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:15.926101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:15.926065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:16.015941 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:16.015910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:17.018034 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018005    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018375    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231806fe-906a-427b-b441-5e31e2c48803" containerName="maas-api"
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018391    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="231806fe-906a-427b-b441-5e31e2c48803" containerName="maas-api"
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018404    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" containerName="payload-processing"
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018413    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" containerName="payload-processing"
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018427    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="111c16ae-1045-471f-aa7a-9a02077f4d49" containerName="payload-pre-processing"
Jul 29 21:33:17.018490 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018433    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="111c16ae-1045-471f-aa7a-9a02077f4d49" containerName="payload-pre-processing"
Jul 29 21:33:17.018806 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018504    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="231806fe-906a-427b-b441-5e31e2c48803" containerName="maas-api"
Jul 29 21:33:17.018806 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018513    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb3e4a6-a3b0-4bae-bb9b-170388f35208" containerName="payload-processing"
Jul 29 21:33:17.018806 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.018522    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="111c16ae-1045-471f-aa7a-9a02077f4d49" containerName="payload-pre-processing"
Jul 29 21:33:17.020304 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.020287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.023338 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.023312    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:33:17.031704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.031687    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:33:17.031865 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.031849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fd48e5e0-serving-cert\""
Jul 29 21:33:17.032413 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.032395    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:33:17.042861 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.042842    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:33:17.080386 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.080354    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:17.099610 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.099582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.099757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.099624    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vbv\" (UniqueName: \"kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.099757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.099649    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.201040 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.201016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.201151 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.201089    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.201151 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.201124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vbv\" (UniqueName: \"kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.201275 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:33:17.201148    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-serving-cert: secret "maas-api-e2e-mt-fd48e5e0-serving-cert" not found
Jul 29 21:33:17.201275 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:33:17.201217    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls podName:380896cf-29ce-430a-a8c9-5b171716cc0a nodeName:}" failed. No retries permitted until 2026-07-29 21:33:17.701201402 +0000 UTC m=+2810.299177742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls") pod "maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" (UID: "380896cf-29ce-430a-a8c9-5b171716cc0a") : secret "maas-api-e2e-mt-fd48e5e0-serving-cert" not found
Jul 29 21:33:17.201686 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.201668    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.221916 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.221890    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vbv\" (UniqueName: \"kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.581912 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.581876    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:33:17.608906 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.608868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:17.700767 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.700719    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:17.705892 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.705869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:17.706020 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:33:17.706005    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-serving-cert: secret "maas-api-e2e-mt-fd48e5e0-serving-cert" not found
Jul 29 21:33:17.706066 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:33:17.706061    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls podName:380896cf-29ce-430a-a8c9-5b171716cc0a nodeName:}" failed. No retries permitted until 2026-07-29 21:33:18.706046467 +0000 UTC m=+2811.304022802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls") pod "maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" (UID: "380896cf-29ce-430a-a8c9-5b171716cc0a") : secret "maas-api-e2e-mt-fd48e5e0-serving-cert" not found
Jul 29 21:33:17.812331 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.812295    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:33:17.814759 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.814727    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:17.817224 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.817201    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fd48e5e0-dockercfg-5zg75\""
Jul 29 21:33:17.817618 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.817605    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fd48e5e0\""
Jul 29 21:33:17.828488 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.828465    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:33:17.852614 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.852529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:17.907262 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.907230    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92nps\" (UniqueName: \"kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:17.907444 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.907269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:17.914215 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.914186    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:33:17.916501 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.916482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:17.935168 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:17.935139    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:33:18.007997 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.007954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.008192 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.008115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.008192 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.008165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92nps\" (UniqueName: \"kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.008321 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.008232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwtx2\" (UniqueName: \"kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.008821 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.008805    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.020146 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.020114    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92nps\" (UniqueName: \"kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps\") pod \"payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.109556 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.109476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwtx2\" (UniqueName: \"kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.109698 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.109581    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.110238 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.110220    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.119129 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.119112    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwtx2\" (UniqueName: \"kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2\") pod \"payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") " pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.125907 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.125880    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.226314 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.226275    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.256279 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.256252    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:33:18.258018 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:33:18.257992    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4ebb43_1d15_4ed4_aba8_8ecdfb8a4bde.slice/crio-5a9313e26cd6c15101fc153c119fce2d52f36abfc938e03feb97a6514dac646c WatchSource:0}: Error finding container 5a9313e26cd6c15101fc153c119fce2d52f36abfc938e03feb97a6514dac646c: Status 404 returned error can't find the container with id 5a9313e26cd6c15101fc153c119fce2d52f36abfc938e03feb97a6514dac646c
Jul 29 21:33:18.359115 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.359093    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:33:18.361340 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:33:18.361289    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729b8a0a_ed3a_45b2_8aec_bfd5cebec895.slice/crio-83417ea28829536a303159f3f5f583d3d0de098ddf8f710d60bb6fe7334fc03b WatchSource:0}: Error finding container 83417ea28829536a303159f3f5f583d3d0de098ddf8f710d60bb6fe7334fc03b: Status 404 returned error can't find the container with id 83417ea28829536a303159f3f5f583d3d0de098ddf8f710d60bb6fe7334fc03b
Jul 29 21:33:18.531859 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.531825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" event={"ID":"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde","Type":"ContainerStarted","Data":"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"}
Jul 29 21:33:18.532034 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.531908    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" event={"ID":"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde","Type":"ContainerStarted","Data":"5a9313e26cd6c15101fc153c119fce2d52f36abfc938e03feb97a6514dac646c"}
Jul 29 21:33:18.532034 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.531942    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:18.533214 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.533193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" event={"ID":"729b8a0a-ed3a-45b2-8aec-bfd5cebec895","Type":"ContainerStarted","Data":"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"}
Jul 29 21:33:18.533273 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.533232    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" event={"ID":"729b8a0a-ed3a-45b2-8aec-bfd5cebec895","Type":"ContainerStarted","Data":"83417ea28829536a303159f3f5f583d3d0de098ddf8f710d60bb6fe7334fc03b"}
Jul 29 21:33:18.533273 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.533266    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:18.552485 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.552447    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" podStartSLOduration=1.552435332 podStartE2EDuration="1.552435332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:33:18.551697655 +0000 UTC m=+2811.149674013" watchObservedRunningTime="2026-07-29 21:33:18.552435332 +0000 UTC m=+2811.150411690"
Jul 29 21:33:18.569841 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.569805    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" podStartSLOduration=1.5697928810000001 podStartE2EDuration="1.569792881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:33:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:33:18.568317462 +0000 UTC m=+2811.166293829" watchObservedRunningTime="2026-07-29 21:33:18.569792881 +0000 UTC m=+2811.167769238"
Jul 29 21:33:18.715794 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.715710    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:18.718044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.718017    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:18.831169 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.831145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:18.967505 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:18.967420    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:33:18.971651 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:33:18.971610    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380896cf_29ce_430a_a8c9_5b171716cc0a.slice/crio-9ee7aeb25e0078c6833533cae0d8112992a2f9f56f0174242a95b249eca6ee66 WatchSource:0}: Error finding container 9ee7aeb25e0078c6833533cae0d8112992a2f9f56f0174242a95b249eca6ee66: Status 404 returned error can't find the container with id 9ee7aeb25e0078c6833533cae0d8112992a2f9f56f0174242a95b249eca6ee66
Jul 29 21:33:19.538326 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:19.538297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" event={"ID":"380896cf-29ce-430a-a8c9-5b171716cc0a","Type":"ContainerStarted","Data":"9ee7aeb25e0078c6833533cae0d8112992a2f9f56f0174242a95b249eca6ee66"}
Jul 29 21:33:20.543939 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:20.543900    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" event={"ID":"380896cf-29ce-430a-a8c9-5b171716cc0a","Type":"ContainerStarted","Data":"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"}
Jul 29 21:33:20.544287 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:20.543968    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:20.560022 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:20.559974    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" podStartSLOduration=3.853435795 podStartE2EDuration="4.559960146s" totalImagesPullingTime="706.524351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:33:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:33:20.558938772 +0000 UTC m=+2813.156915152" watchObservedRunningTime="2026-07-29 21:33:20.559960146 +0000 UTC m=+2813.157936513"
Jul 29 21:33:26.553919 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:26.553885    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:33:29.540404 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:29.540368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:33:29.540860 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:29.540662    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:33:30.257816 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:30.257778    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:33:40.947591 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:40.947554    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:44.280039 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:44.280004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:47.861959 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:47.861926    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:33:54.643357 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:54.643316    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:54.726582 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:33:54.726538    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:08.266309 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:08.266268    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:34:34.760036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:34.759950    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:34:36.044518 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.044480    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:34:36.044956 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.044716    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" podUID="380896cf-29ce-430a-a8c9-5b171716cc0a" containerName="maas-api" containerID="cri-o://feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f" gracePeriod=30
Jul 29 21:34:36.067806 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.067778    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:36.092874 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.092849    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:34:36.093047 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.093024    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" podUID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" containerName="payload-processing" containerID="cri-o://22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183" gracePeriod=30
Jul 29 21:34:36.101620 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.101595    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:34:36.101845 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.101818    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" podUID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" containerName="payload-pre-processing" containerID="cri-o://3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b" gracePeriod=30
Jul 29 21:34:36.166527 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.166461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:36.243249 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.241792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:36.300584 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.300490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:36.357255 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.357231    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:34:36.389221 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.389198    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:34:36.393915 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.393897    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:34:36.424224 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424200    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca\") pod \"380896cf-29ce-430a-a8c9-5b171716cc0a\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") "
Jul 29 21:34:36.424342 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2vbv\" (UniqueName: \"kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv\") pod \"380896cf-29ce-430a-a8c9-5b171716cc0a\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") "
Jul 29 21:34:36.424342 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424265    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwtx2\" (UniqueName: \"kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2\") pod \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") "
Jul 29 21:34:36.424342 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424291    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") pod \"380896cf-29ce-430a-a8c9-5b171716cc0a\" (UID: \"380896cf-29ce-430a-a8c9-5b171716cc0a\") "
Jul 29 21:34:36.424521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424438    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume\") pod \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\" (UID: \"729b8a0a-ed3a-45b2-8aec-bfd5cebec895\") "
Jul 29 21:34:36.424521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424484    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92nps\" (UniqueName: \"kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps\") pod \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") "
Jul 29 21:34:36.424521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424516    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume\") pod \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\" (UID: \"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde\") "
Jul 29 21:34:36.424684 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424658    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "380896cf-29ce-430a-a8c9-5b171716cc0a" (UID: "380896cf-29ce-430a-a8c9-5b171716cc0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:34:36.424949 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.424914    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/380896cf-29ce-430a-a8c9-5b171716cc0a-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.425329 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.425302    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" (UID: "cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:34:36.425432 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.425318    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "729b8a0a-ed3a-45b2-8aec-bfd5cebec895" (UID: "729b8a0a-ed3a-45b2-8aec-bfd5cebec895"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:34:36.427303 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.427275    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2" (OuterVolumeSpecName: "kube-api-access-dwtx2") pod "729b8a0a-ed3a-45b2-8aec-bfd5cebec895" (UID: "729b8a0a-ed3a-45b2-8aec-bfd5cebec895"). InnerVolumeSpecName "kube-api-access-dwtx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:34:36.427507 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.427484    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "380896cf-29ce-430a-a8c9-5b171716cc0a" (UID: "380896cf-29ce-430a-a8c9-5b171716cc0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:34:36.427967 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.427945    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv" (OuterVolumeSpecName: "kube-api-access-l2vbv") pod "380896cf-29ce-430a-a8c9-5b171716cc0a" (UID: "380896cf-29ce-430a-a8c9-5b171716cc0a"). InnerVolumeSpecName "kube-api-access-l2vbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:34:36.428212 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.428188    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps" (OuterVolumeSpecName: "kube-api-access-92nps") pod "cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" (UID: "cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde"). InnerVolumeSpecName "kube-api-access-92nps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:34:36.525566 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2vbv\" (UniqueName: \"kubernetes.io/projected/380896cf-29ce-430a-a8c9-5b171716cc0a-kube-api-access-l2vbv\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.525566 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525567    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwtx2\" (UniqueName: \"kubernetes.io/projected/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-kube-api-access-dwtx2\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.525704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525576    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/380896cf-29ce-430a-a8c9-5b171716cc0a-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.525704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525585    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/729b8a0a-ed3a-45b2-8aec-bfd5cebec895-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.525704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525594    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92nps\" (UniqueName: \"kubernetes.io/projected/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-kube-api-access-92nps\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.525704 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.525603    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:34:36.915795 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.915757    2569 generic.go:356] "Generic (PLEG): container finished" podID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" containerID="3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b" exitCode=0
Jul 29 21:34:36.915950 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.915806    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"
Jul 29 21:34:36.915950 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.915828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" event={"ID":"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde","Type":"ContainerDied","Data":"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"}
Jul 29 21:34:36.915950 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.915887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw" event={"ID":"cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde","Type":"ContainerDied","Data":"5a9313e26cd6c15101fc153c119fce2d52f36abfc938e03feb97a6514dac646c"}
Jul 29 21:34:36.915950 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.915916    2569 scope.go:117] "RemoveContainer" containerID="3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"
Jul 29 21:34:36.917400 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.917374    2569 generic.go:356] "Generic (PLEG): container finished" podID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" containerID="22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183" exitCode=0
Jul 29 21:34:36.917521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.917421    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"
Jul 29 21:34:36.917521 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.917465    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" event={"ID":"729b8a0a-ed3a-45b2-8aec-bfd5cebec895","Type":"ContainerDied","Data":"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"}
Jul 29 21:34:36.917641 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.917538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd" event={"ID":"729b8a0a-ed3a-45b2-8aec-bfd5cebec895","Type":"ContainerDied","Data":"83417ea28829536a303159f3f5f583d3d0de098ddf8f710d60bb6fe7334fc03b"}
Jul 29 21:34:36.919439 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.919369    2569 generic.go:356] "Generic (PLEG): container finished" podID="380896cf-29ce-430a-a8c9-5b171716cc0a" containerID="feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f" exitCode=0
Jul 29 21:34:36.919439 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.919422    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"
Jul 29 21:34:36.919439 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.919426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" event={"ID":"380896cf-29ce-430a-a8c9-5b171716cc0a","Type":"ContainerDied","Data":"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"}
Jul 29 21:34:36.919684 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.919468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh" event={"ID":"380896cf-29ce-430a-a8c9-5b171716cc0a","Type":"ContainerDied","Data":"9ee7aeb25e0078c6833533cae0d8112992a2f9f56f0174242a95b249eca6ee66"}
Jul 29 21:34:36.928244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.928200    2569 scope.go:117] "RemoveContainer" containerID="3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"
Jul 29 21:34:36.928540 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:34:36.928517    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b\": container with ID starting with 3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b not found: ID does not exist" containerID="3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"
Jul 29 21:34:36.928621 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.928546    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b"} err="failed to get container status \"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b\": rpc error: code = NotFound desc = could not find container \"3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b\": container with ID starting with 3c953d104a6b018f76f25f04e85476f25e09a3752cefb707f00decb6660a518b not found: ID does not exist"
Jul 29 21:34:36.928621 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.928565    2569 scope.go:117] "RemoveContainer" containerID="22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"
Jul 29 21:34:36.937297 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.937280    2569 scope.go:117] "RemoveContainer" containerID="22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"
Jul 29 21:34:36.937618 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:34:36.937593    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183\": container with ID starting with 22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183 not found: ID does not exist" containerID="22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"
Jul 29 21:34:36.937780 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.937619    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183"} err="failed to get container status \"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183\": rpc error: code = NotFound desc = could not find container \"22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183\": container with ID starting with 22d0a45c258546eb7510a59e075d77cf4ceb5ef4a23abd1c43ab9377be32b183 not found: ID does not exist"
Jul 29 21:34:36.937780 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.937637    2569 scope.go:117] "RemoveContainer" containerID="feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"
Jul 29 21:34:36.943703 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.943676    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:34:36.945844 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.945824    2569 scope.go:117] "RemoveContainer" containerID="feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"
Jul 29 21:34:36.946112 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:34:36.946092    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f\": container with ID starting with feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f not found: ID does not exist" containerID="feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"
Jul 29 21:34:36.946193 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.946114    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f"} err="failed to get container status \"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f\": rpc error: code = NotFound desc = could not find container \"feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f\": container with ID starting with feaf0badf74fe394b59ee5fc60be612c4248aa2d490cecd26319ea293257329f not found: ID does not exist"
Jul 29 21:34:36.947223 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.947198    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-fd48e5e0-868fc85748-66fgw"]
Jul 29 21:34:36.957251 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.957228    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:34:36.960966 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.960943    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fd48e5e0-6c5778fb94-2qzwd"]
Jul 29 21:34:36.971345 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.971322    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:34:36.976123 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:36.976095    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fd48e5e0-55d9f949f9-jvbsh"]
Jul 29 21:34:37.898669 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:37.898634    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="380896cf-29ce-430a-a8c9-5b171716cc0a" path="/var/lib/kubelet/pods/380896cf-29ce-430a-a8c9-5b171716cc0a/volumes"
Jul 29 21:34:37.899072 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:37.899021    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" path="/var/lib/kubelet/pods/729b8a0a-ed3a-45b2-8aec-bfd5cebec895/volumes"
Jul 29 21:34:37.899339 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:37.899325    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" path="/var/lib/kubelet/pods/cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde/volumes"
Jul 29 21:34:39.353597 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:39.353544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:44.054959 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:44.054929    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:34:44.425569 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:34:44.425476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:01.322887 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.322849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:01.493227 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.493189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:01.511532 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.511500    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:35:01.511707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.511688    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" podUID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" containerName="payload-processing" containerID="cri-o://f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8" gracePeriod=30
Jul 29 21:35:01.518339 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.518311    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:35:01.518558 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.518518    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" podUID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" containerName="payload-pre-processing" containerID="cri-o://00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a" gracePeriod=30
Jul 29 21:35:01.632445 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.632416    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:01.753900 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.753879    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:35:01.783216 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.783192    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:35:01.813293 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.813261    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume\") pod \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") "
Jul 29 21:35:01.813428 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.813394    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgs65\" (UniqueName: \"kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65\") pod \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\" (UID: \"b0fa73f6-63e7-4966-87b2-01ced9ce217c\") "
Jul 29 21:35:01.813642 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.813619    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0fa73f6-63e7-4966-87b2-01ced9ce217c" (UID: "b0fa73f6-63e7-4966-87b2-01ced9ce217c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:35:01.815382 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.815362    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65" (OuterVolumeSpecName: "kube-api-access-bgs65") pod "b0fa73f6-63e7-4966-87b2-01ced9ce217c" (UID: "b0fa73f6-63e7-4966-87b2-01ced9ce217c"). InnerVolumeSpecName "kube-api-access-bgs65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:35:01.914243 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.914209    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8krc\" (UniqueName: \"kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc\") pod \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") "
Jul 29 21:35:01.914364 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.914336    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume\") pod \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\" (UID: \"00c896ce-75d5-4e8c-bd51-deec53fab8c3\") "
Jul 29 21:35:01.914518 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.914504    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgs65\" (UniqueName: \"kubernetes.io/projected/b0fa73f6-63e7-4966-87b2-01ced9ce217c-kube-api-access-bgs65\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:35:01.914569 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.914519    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0fa73f6-63e7-4966-87b2-01ced9ce217c-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:35:01.914717 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.914697    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00c896ce-75d5-4e8c-bd51-deec53fab8c3" (UID: "00c896ce-75d5-4e8c-bd51-deec53fab8c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:35:01.916176 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:01.916152    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc" (OuterVolumeSpecName: "kube-api-access-w8krc") pod "00c896ce-75d5-4e8c-bd51-deec53fab8c3" (UID: "00c896ce-75d5-4e8c-bd51-deec53fab8c3"). InnerVolumeSpecName "kube-api-access-w8krc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:35:02.015703 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.015677    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c896ce-75d5-4e8c-bd51-deec53fab8c3-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:35:02.015703 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.015704    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8krc\" (UniqueName: \"kubernetes.io/projected/00c896ce-75d5-4e8c-bd51-deec53fab8c3-kube-api-access-w8krc\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:35:02.030846 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.030817    2569 generic.go:356] "Generic (PLEG): container finished" podID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" containerID="00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a" exitCode=0
Jul 29 21:35:02.030948 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.030876    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"
Jul 29 21:35:02.030948 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.030904    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" event={"ID":"00c896ce-75d5-4e8c-bd51-deec53fab8c3","Type":"ContainerDied","Data":"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"}
Jul 29 21:35:02.031116 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.030962    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr" event={"ID":"00c896ce-75d5-4e8c-bd51-deec53fab8c3","Type":"ContainerDied","Data":"87e029e61a02813501742d77107f76058d48ebbdbacb470c677c4302359284de"}
Jul 29 21:35:02.031116 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.030993    2569 scope.go:117] "RemoveContainer" containerID="00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"
Jul 29 21:35:02.032194 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.032168    2569 generic.go:356] "Generic (PLEG): container finished" podID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" containerID="f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8" exitCode=0
Jul 29 21:35:02.032277 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.032193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" event={"ID":"b0fa73f6-63e7-4966-87b2-01ced9ce217c","Type":"ContainerDied","Data":"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"}
Jul 29 21:35:02.032277 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.032222    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"
Jul 29 21:35:02.032277 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.032237    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5" event={"ID":"b0fa73f6-63e7-4966-87b2-01ced9ce217c","Type":"ContainerDied","Data":"4d029286d928f836b60f586f7b7767cb1bfc0e5ce1821addebe50bdde2451f3c"}
Jul 29 21:35:02.039848 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.039834    2569 scope.go:117] "RemoveContainer" containerID="00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"
Jul 29 21:35:02.040154 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:35:02.040134    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a\": container with ID starting with 00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a not found: ID does not exist" containerID="00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"
Jul 29 21:35:02.040244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.040163    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a"} err="failed to get container status \"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a\": rpc error: code = NotFound desc = could not find container \"00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a\": container with ID starting with 00984951fd1dd64a23b365dd5fe9cc38516e27992d17004360d0d73a9c3b825a not found: ID does not exist"
Jul 29 21:35:02.040244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.040186    2569 scope.go:117] "RemoveContainer" containerID="f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"
Jul 29 21:35:02.041111 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.040997    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:02.047500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.047478    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:35:02.047702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.047685    2569 scope.go:117] "RemoveContainer" containerID="f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"
Jul 29 21:35:02.047962 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:35:02.047946    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8\": container with ID starting with f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8 not found: ID does not exist" containerID="f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"
Jul 29 21:35:02.048025 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.047968    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8"} err="failed to get container status \"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8\": rpc error: code = NotFound desc = could not find container \"f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8\": container with ID starting with f2b5edd02e060aea2d57e3c0cf05714dfa88fc2e6ef9cacfc6d5ae6271bb22f8 not found: ID does not exist"
Jul 29 21:35:02.055101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.055081    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f7c67e2c-76f54f94f8-sxgj5"]
Jul 29 21:35:02.065715 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.065687    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:35:02.067322 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.067303    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f7c67e2c-5dfb585f64-7v4vr"]
Jul 29 21:35:02.112893 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:02.112872    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:03.898436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:03.898404    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" path="/var/lib/kubelet/pods/00c896ce-75d5-4e8c-bd51-deec53fab8c3/volumes"
Jul 29 21:35:03.898807 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:03.898761    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" path="/var/lib/kubelet/pods/b0fa73f6-63e7-4966-87b2-01ced9ce217c/volumes"
Jul 29 21:35:21.131469 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:21.131435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:21.466486 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:21.466382    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:27.070203 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:27.070158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:27.130745 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:27.130714    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:27.342670 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:27.342589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:27.590377 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:27.590351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:30.054406 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:30.054374    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:35:39.092537 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:39.092502    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:39.487313 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:39.487279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:39.679974 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:39.679944    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:42.049712 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:42.049684    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:35:42.572100 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:42.572066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:42.670018 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:42.669980    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:43.646108 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646076    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:35:43.646572 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646556    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" containerName="payload-pre-processing"
Jul 29 21:35:43.646625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646576    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" containerName="payload-pre-processing"
Jul 29 21:35:43.646625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646609    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="380896cf-29ce-430a-a8c9-5b171716cc0a" containerName="maas-api"
Jul 29 21:35:43.646625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646618    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="380896cf-29ce-430a-a8c9-5b171716cc0a" containerName="maas-api"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646632    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" containerName="payload-pre-processing"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646641    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" containerName="payload-pre-processing"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646651    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" containerName="payload-processing"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646659    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" containerName="payload-processing"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646677    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" containerName="payload-processing"
Jul 29 21:35:43.646764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646685    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" containerName="payload-processing"
Jul 29 21:35:43.646954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646877    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="00c896ce-75d5-4e8c-bd51-deec53fab8c3" containerName="payload-pre-processing"
Jul 29 21:35:43.646954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646891    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="729b8a0a-ed3a-45b2-8aec-bfd5cebec895" containerName="payload-processing"
Jul 29 21:35:43.646954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646905    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0fa73f6-63e7-4966-87b2-01ced9ce217c" containerName="payload-processing"
Jul 29 21:35:43.646954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646914    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="380896cf-29ce-430a-a8c9-5b171716cc0a" containerName="maas-api"
Jul 29 21:35:43.646954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.646924    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd4ebb43-1d15-4ed4-aba8-8ecdfb8a4bde" containerName="payload-pre-processing"
Jul 29 21:35:43.650409 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.650388    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.653114 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.653090    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:35:43.653249 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.653127    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-bdcc5a3e-serving-cert\""
Jul 29 21:35:43.654001 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.653985    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:35:43.654078 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.653999    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:35:43.659199 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.659175    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:35:43.682601 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.682562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:43.822637 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.822602    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm84s\" (UniqueName: \"kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.822808 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.822659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.822808 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.822680    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.891040 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.891007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:43.923096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.923033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm84s\" (UniqueName: \"kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.923096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.923080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.923252 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.923103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.923644 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.923620    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.925436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.925418    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.931006 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.930988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm84s\" (UniqueName: \"kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s\") pod \"maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:43.963003 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:43.962981    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:44.092774 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.092747    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:35:44.096058 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:35:44.096017    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7eeb4bf_f46b_4701_b103_129148b74ad6.slice/crio-791cc5c191ef60e37e92b7b5666b40e73db949ce87214f05a128a3add1b99e49 WatchSource:0}: Error finding container 791cc5c191ef60e37e92b7b5666b40e73db949ce87214f05a128a3add1b99e49: Status 404 returned error can't find the container with id 791cc5c191ef60e37e92b7b5666b40e73db949ce87214f05a128a3add1b99e49
Jul 29 21:35:44.098109 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.098092    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:35:44.186398 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.186329    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:44.215434 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.215401    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" event={"ID":"a7eeb4bf-f46b-4701-b103-129148b74ad6","Type":"ContainerStarted","Data":"791cc5c191ef60e37e92b7b5666b40e73db949ce87214f05a128a3add1b99e49"}
Jul 29 21:35:44.287430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.287397    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:35:44.288411 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.288387    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:35:44.292848 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.292830    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.295379 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.295361    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bdcc5a3e-dockercfg-rs7rp\""
Jul 29 21:35:44.296545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.296526    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bdcc5a3e\""
Jul 29 21:35:44.302622 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.302598    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:35:44.315193 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.315170    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:35:44.319276 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.319255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.327044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.327023    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:35:44.427465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.427435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5dql\" (UniqueName: \"kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.427615 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.427470    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.427615 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.427488    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gtsv\" (UniqueName: \"kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.427759 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.427644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.528471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.528448    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.528605 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.528493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5dql\" (UniqueName: \"kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.528605 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.528516    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.528605 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.528534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gtsv\" (UniqueName: \"kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.529236 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.529212    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.529236 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.529225    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.536462 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.536438    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gtsv\" (UniqueName: \"kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv\") pod \"payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.536559 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.536439    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5dql\" (UniqueName: \"kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql\") pod \"payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") " pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.604227 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.604198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:44.630916 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.630891    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:44.754199 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.753913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:35:44.756349 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:35:44.756319    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bb4a35_06ac_48c9_8c48_79f747232c85.slice/crio-1a9d2a53445882fc172e6215fe933ae5887989f5ceb41dd980a6ddb72546804e WatchSource:0}: Error finding container 1a9d2a53445882fc172e6215fe933ae5887989f5ceb41dd980a6ddb72546804e: Status 404 returned error can't find the container with id 1a9d2a53445882fc172e6215fe933ae5887989f5ceb41dd980a6ddb72546804e
Jul 29 21:35:44.781119 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:44.781096    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:35:44.788042 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:35:44.788019    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05d4497_3b3f_4dd1_87f7_f36dccdb6768.slice/crio-1c200fa03891f4745a776d488b382e8210027380175594e7512f0d2135c4cb78 WatchSource:0}: Error finding container 1c200fa03891f4745a776d488b382e8210027380175594e7512f0d2135c4cb78: Status 404 returned error can't find the container with id 1c200fa03891f4745a776d488b382e8210027380175594e7512f0d2135c4cb78
Jul 29 21:35:45.222590 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.222492    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" event={"ID":"f05d4497-3b3f-4dd1-87f7-f36dccdb6768","Type":"ContainerStarted","Data":"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"}
Jul 29 21:35:45.222590 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.222561    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" event={"ID":"f05d4497-3b3f-4dd1-87f7-f36dccdb6768","Type":"ContainerStarted","Data":"1c200fa03891f4745a776d488b382e8210027380175594e7512f0d2135c4cb78"}
Jul 29 21:35:45.222844 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.222594    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:45.224012 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.223992    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" event={"ID":"a7eeb4bf-f46b-4701-b103-129148b74ad6","Type":"ContainerStarted","Data":"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"}
Jul 29 21:35:45.224104 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.224034    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:45.225367 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.225343    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" event={"ID":"f6bb4a35-06ac-48c9-8c48-79f747232c85","Type":"ContainerStarted","Data":"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"}
Jul 29 21:35:45.225477 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.225385    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:35:45.225477 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.225402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" event={"ID":"f6bb4a35-06ac-48c9-8c48-79f747232c85","Type":"ContainerStarted","Data":"1a9d2a53445882fc172e6215fe933ae5887989f5ceb41dd980a6ddb72546804e"}
Jul 29 21:35:45.241300 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.241261    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" podStartSLOduration=1.241247127 podStartE2EDuration="1.241247127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:35:45.239903292 +0000 UTC m=+2957.837879650" watchObservedRunningTime="2026-07-29 21:35:45.241247127 +0000 UTC m=+2957.839223485"
Jul 29 21:35:45.256700 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.256660    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" podStartSLOduration=1.256648048 podStartE2EDuration="1.256648048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:35:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:35:45.255332089 +0000 UTC m=+2957.853308447" watchObservedRunningTime="2026-07-29 21:35:45.256648048 +0000 UTC m=+2957.854624405"
Jul 29 21:35:45.272332 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:45.272293    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" podStartSLOduration=1.7585526169999999 podStartE2EDuration="2.272281996s" totalImagesPullingTime="513.729379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:35:45.270352725 +0000 UTC m=+2957.868329101" watchObservedRunningTime="2026-07-29 21:35:45.272281996 +0000 UTC m=+2957.870258352"
Jul 29 21:35:51.235634 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:51.235558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:35:52.350826 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:52.350787    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:35:56.231863 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:56.231824    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:35:56.232307 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:35:56.232173    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:36:07.536204 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:07.536171    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:12.986506 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:12.986464    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:18.672228 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:18.672189    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:18.950295 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:18.950211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:19.640268 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:19.640234    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:21.669294 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:21.669252    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:21.773482 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:21.773449    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:22.766850 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.766821    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:36:22.770640 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.770619    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.773224 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.773205    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-641c4002-serving-cert\""
Jul 29 21:36:22.783367 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.783224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:36:22.807006 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.806971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:22.824946 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.824918    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.825052 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.824969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz2r4\" (UniqueName: \"kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.825116 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.825049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.925727 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.925695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.925897 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.925759    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.925897 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.925798    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz2r4\" (UniqueName: \"kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.926017 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:36:22.925903    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-serving-cert: secret "maas-api-e2e-mt-641c4002-serving-cert" not found
Jul 29 21:36:22.926017 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:36:22.925972    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls podName:d2e50466-c763-4b4e-ba71-e2e3fcf53c4d nodeName:}" failed. No retries permitted until 2026-07-29 21:36:23.425956316 +0000 UTC m=+2996.023932650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls") pod "maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" (UID: "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d") : secret "maas-api-e2e-mt-641c4002-serving-cert" not found
Jul 29 21:36:22.926332 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.926314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:22.934126 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:22.934107    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz2r4\" (UniqueName: \"kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:23.035830 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.035730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:23.430587 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.430505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:23.432899 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.432878    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") pod \"maas-api-e2e-mt-641c4002-8544dbf754-gvkb2\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:23.438398 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.438365    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:23.542747 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.542700    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:36:23.544934 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.544908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:23.546384 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.546369    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.548820 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.548800    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-641c4002-dockercfg-7rs9r\""
Jul 29 21:36:23.548927 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.548825    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-641c4002\""
Jul 29 21:36:23.557111 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.557089    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:36:23.632128 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.632097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ml8rm\" (UniqueName: \"kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.632286 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.632134    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.683672 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.683590    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:23.733658 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.733615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ml8rm\" (UniqueName: \"kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.733796 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.733691    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.734670 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.734643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.742243 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.742214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ml8rm\" (UniqueName: \"kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm\") pod \"payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.857391 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.857357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:23.993970 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:23.991378    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:36:24.024687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.024663    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:36:24.406077 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.406042    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" event={"ID":"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d","Type":"ContainerStarted","Data":"a21827801b070df7ccb8639233ab641ef998801cc3bda3635f4085eeeb919032"}
Jul 29 21:36:24.407465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.407438    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" event={"ID":"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656","Type":"ContainerStarted","Data":"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"}
Jul 29 21:36:24.407600 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.407479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" event={"ID":"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656","Type":"ContainerStarted","Data":"65944f18c2a1368507280445aeb614b111ac64d6df8eb619483e94f38ef2f7ff"}
Jul 29 21:36:24.407600 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.407586    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:24.423248 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:24.423205    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" podStartSLOduration=1.423191603 podStartE2EDuration="1.423191603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:36:24.422000595 +0000 UTC m=+2997.019976972" watchObservedRunningTime="2026-07-29 21:36:24.423191603 +0000 UTC m=+2997.021167960"
Jul 29 21:36:25.415040 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:25.415004    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" event={"ID":"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d","Type":"ContainerStarted","Data":"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"}
Jul 29 21:36:25.415514 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:25.415351    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:25.435147 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:25.435098    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" podStartSLOduration=2.8889255990000002 podStartE2EDuration="3.435085017s" totalImagesPullingTime="546.159418ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:36:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:36:25.433908719 +0000 UTC m=+2998.031885077" watchObservedRunningTime="2026-07-29 21:36:25.435085017 +0000 UTC m=+2998.033061374"
Jul 29 21:36:31.060221 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:31.060185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:36:31.425181 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:31.425107    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:36:35.416528 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:35.416481    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.50:9004: connect: connection refused"
Jul 29 21:36:45.416049 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:45.416015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:36:46.639227 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:46.639192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:49.317855 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:49.317818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:36:49.418926 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:36:49.418887    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:37:22.612515 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:37:22.612385    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:03.416068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:43:03.416027    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:32.378608 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:43:32.378534    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:38.190447 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:43:38.190410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:46.181824 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:43:46.181782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:46.240565 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:43:46.240532    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.643030 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:02.642953    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:02.870707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:02.870678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:06.049177 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:06.049144    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:45:18.450434 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.450399    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:45:18.450878 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.450667    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" podUID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" containerName="maas-api" containerID="cri-o://0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1" gracePeriod=30
Jul 29 21:45:18.490897 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.490870    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:18.529348 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.529320    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:45:18.529574 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.529541    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerName="payload-pre-processing" containerID="cri-o://ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907" gracePeriod=30
Jul 29 21:45:18.590826 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.590800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:18.690405 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.690360    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:18.773021 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.772999    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:45:18.796101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.796065    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:18.800065 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.800045    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:45:18.859074 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859049    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca\") pod \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") "
Jul 29 21:45:18.859173 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859106    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") pod \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") "
Jul 29 21:45:18.859316 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859296    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz2r4\" (UniqueName: \"kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4\") pod \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\" (UID: \"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d\") "
Jul 29 21:45:18.859372 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume\") pod \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") "
Jul 29 21:45:18.859372 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859359    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ml8rm\" (UniqueName: \"kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm\") pod \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\" (UID: \"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656\") "
Jul 29 21:45:18.859481 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859373    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" (UID: "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:18.859647 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859624    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:45:18.859775 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.859642    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" (UID: "a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:45:18.861078 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.861057    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" (UID: "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:45:18.861264 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.861243    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm" (OuterVolumeSpecName: "kube-api-access-ml8rm") pod "a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" (UID: "a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656"). InnerVolumeSpecName "kube-api-access-ml8rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:18.861317 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.861265    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4" (OuterVolumeSpecName: "kube-api-access-vz2r4") pod "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" (UID: "d2e50466-c763-4b4e-ba71-e2e3fcf53c4d"). InnerVolumeSpecName "kube-api-access-vz2r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:45:18.902357 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.902338    2569 generic.go:356] "Generic (PLEG): container finished" podID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerID="ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907" exitCode=0
Jul 29 21:45:18.902452 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.902389    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"
Jul 29 21:45:18.902452 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.902398    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" event={"ID":"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656","Type":"ContainerDied","Data":"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"}
Jul 29 21:45:18.902452 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.902443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc" event={"ID":"a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656","Type":"ContainerDied","Data":"65944f18c2a1368507280445aeb614b111ac64d6df8eb619483e94f38ef2f7ff"}
Jul 29 21:45:18.902593 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.902468    2569 scope.go:117] "RemoveContainer" containerID="ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"
Jul 29 21:45:18.903655 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.903631    2569 generic.go:356] "Generic (PLEG): container finished" podID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" containerID="0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1" exitCode=0
Jul 29 21:45:18.903724 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.903661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" event={"ID":"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d","Type":"ContainerDied","Data":"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"}
Jul 29 21:45:18.903724 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.903689    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"
Jul 29 21:45:18.903724 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.903704    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2" event={"ID":"d2e50466-c763-4b4e-ba71-e2e3fcf53c4d","Type":"ContainerDied","Data":"a21827801b070df7ccb8639233ab641ef998801cc3bda3635f4085eeeb919032"}
Jul 29 21:45:18.911801 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.911786    2569 scope.go:117] "RemoveContainer" containerID="ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"
Jul 29 21:45:18.912045 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:45:18.912025    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907\": container with ID starting with ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907 not found: ID does not exist" containerID="ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"
Jul 29 21:45:18.912174 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.912054    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907"} err="failed to get container status \"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907\": rpc error: code = NotFound desc = could not find container \"ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907\": container with ID starting with ad9ffac725f1bb904da16cc27de9758d3478b7eb983f5054d2d7523ba21ac907 not found: ID does not exist"
Jul 29 21:45:18.912174 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.912077    2569 scope.go:117] "RemoveContainer" containerID="0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"
Jul 29 21:45:18.919620 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.919602    2569 scope.go:117] "RemoveContainer" containerID="0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"
Jul 29 21:45:18.919881 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:45:18.919862    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1\": container with ID starting with 0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1 not found: ID does not exist" containerID="0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"
Jul 29 21:45:18.919940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.919887    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1"} err="failed to get container status \"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1\": rpc error: code = NotFound desc = could not find container \"0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1\": container with ID starting with 0d8b6750137376ce3b94d92078236b91a424a1b004afb8b73e3af2b5f04cf5c1 not found: ID does not exist"
Jul 29 21:45:18.923281 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.923260    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:45:18.927500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.927478    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-641c4002-795f45b7dd-xd5bc"]
Jul 29 21:45:18.937835 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.937815    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:45:18.941773 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.941752    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-641c4002-8544dbf754-gvkb2"]
Jul 29 21:45:18.960007 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.959984    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:45:18.960007 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.960004    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz2r4\" (UniqueName: \"kubernetes.io/projected/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d-kube-api-access-vz2r4\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:45:18.960123 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.960018    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:45:18.960123 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:18.960030    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ml8rm\" (UniqueName: \"kubernetes.io/projected/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656-kube-api-access-ml8rm\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:45:19.079394 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:19.079339    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:19.897377 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:19.897348    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" path="/var/lib/kubelet/pods/a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656/volumes"
Jul 29 21:45:19.897708 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:19.897694    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" path="/var/lib/kubelet/pods/d2e50466-c763-4b4e-ba71-e2e3fcf53c4d/volumes"
Jul 29 21:45:20.938278 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:20.938249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:23.749378 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:23.749345    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:45:25.097940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.097884    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:25.163206 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.163180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:25.374408 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.374333    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:25.612334 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.612307    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:25.761686 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.761654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:25.867764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:25.867723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:36.395877 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:36.395814    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:36.624589 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:36.624559    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:57.570807 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:45:57.570772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:01.082303 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.082272    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:46:01.082661 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.082473    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerName="maas-api" containerID="cri-o://ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc" gracePeriod=30
Jul 29 21:46:01.109995 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.109972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:01.143060 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.143028    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:46:01.143282 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.143257    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" podUID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" containerName="payload-processing" containerID="cri-o://3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0" gracePeriod=30
Jul 29 21:46:01.149046 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.149022    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:46:01.149203 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.149163    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" podUID="f6bb4a35-06ac-48c9-8c48-79f747232c85" containerName="payload-pre-processing" containerID="cri-o://92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d" gracePeriod=30
Jul 29 21:46:01.225054 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.225032    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:01.230642 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.230620    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.46:8443/health\": dial tcp 10.134.0.46:8443: connect: connection refused"
Jul 29 21:46:01.321125 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.321100    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:01.400242 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.400164    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:46:01.423960 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.423930    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:46:01.433952 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.433927    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:46:01.573028 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.572993    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls\") pod \"a7eeb4bf-f46b-4701-b103-129148b74ad6\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") "
Jul 29 21:46:01.573028 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573033    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5dql\" (UniqueName: \"kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql\") pod \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") "
Jul 29 21:46:01.573248 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573069    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume\") pod \"f6bb4a35-06ac-48c9-8c48-79f747232c85\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") "
Jul 29 21:46:01.573248 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573128    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gtsv\" (UniqueName: \"kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv\") pod \"f6bb4a35-06ac-48c9-8c48-79f747232c85\" (UID: \"f6bb4a35-06ac-48c9-8c48-79f747232c85\") "
Jul 29 21:46:01.573248 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573154    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm84s\" (UniqueName: \"kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s\") pod \"a7eeb4bf-f46b-4701-b103-129148b74ad6\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") "
Jul 29 21:46:01.573248 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573181    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca\") pod \"a7eeb4bf-f46b-4701-b103-129148b74ad6\" (UID: \"a7eeb4bf-f46b-4701-b103-129148b74ad6\") "
Jul 29 21:46:01.573455 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573305    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume\") pod \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\" (UID: \"f05d4497-3b3f-4dd1-87f7-f36dccdb6768\") "
Jul 29 21:46:01.573554 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573516    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6bb4a35-06ac-48c9-8c48-79f747232c85" (UID: "f6bb4a35-06ac-48c9-8c48-79f747232c85"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:46:01.573554 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7eeb4bf-f46b-4701-b103-129148b74ad6" (UID: "a7eeb4bf-f46b-4701-b103-129148b74ad6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:46:01.573666 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573646    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb4a35-06ac-48c9-8c48-79f747232c85-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.573719 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573672    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7eeb4bf-f46b-4701-b103-129148b74ad6-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.573953 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.573930    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f05d4497-3b3f-4dd1-87f7-f36dccdb6768" (UID: "f05d4497-3b3f-4dd1-87f7-f36dccdb6768"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:46:01.575260 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.575241    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7eeb4bf-f46b-4701-b103-129148b74ad6" (UID: "a7eeb4bf-f46b-4701-b103-129148b74ad6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:46:01.575324 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.575255    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s" (OuterVolumeSpecName: "kube-api-access-pm84s") pod "a7eeb4bf-f46b-4701-b103-129148b74ad6" (UID: "a7eeb4bf-f46b-4701-b103-129148b74ad6"). InnerVolumeSpecName "kube-api-access-pm84s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:46:01.575555 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.575535    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql" (OuterVolumeSpecName: "kube-api-access-q5dql") pod "f05d4497-3b3f-4dd1-87f7-f36dccdb6768" (UID: "f05d4497-3b3f-4dd1-87f7-f36dccdb6768"). InnerVolumeSpecName "kube-api-access-q5dql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:46:01.575650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.575560    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv" (OuterVolumeSpecName: "kube-api-access-5gtsv") pod "f6bb4a35-06ac-48c9-8c48-79f747232c85" (UID: "f6bb4a35-06ac-48c9-8c48-79f747232c85"). InnerVolumeSpecName "kube-api-access-5gtsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:46:01.674044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.673994    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gtsv\" (UniqueName: \"kubernetes.io/projected/f6bb4a35-06ac-48c9-8c48-79f747232c85-kube-api-access-5gtsv\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.674044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.674013    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm84s\" (UniqueName: \"kubernetes.io/projected/a7eeb4bf-f46b-4701-b103-129148b74ad6-kube-api-access-pm84s\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.674044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.674022    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.674044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.674031    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7eeb4bf-f46b-4701-b103-129148b74ad6-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:01.674044 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:01.674041    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5dql\" (UniqueName: \"kubernetes.io/projected/f05d4497-3b3f-4dd1-87f7-f36dccdb6768-kube-api-access-q5dql\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:46:02.096388 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.096359    2569 generic.go:356] "Generic (PLEG): container finished" podID="f6bb4a35-06ac-48c9-8c48-79f747232c85" containerID="92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d" exitCode=0
Jul 29 21:46:02.096757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.096414    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"
Jul 29 21:46:02.096757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.096468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" event={"ID":"f6bb4a35-06ac-48c9-8c48-79f747232c85","Type":"ContainerDied","Data":"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"}
Jul 29 21:46:02.096757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.096550    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92" event={"ID":"f6bb4a35-06ac-48c9-8c48-79f747232c85","Type":"ContainerDied","Data":"1a9d2a53445882fc172e6215fe933ae5887989f5ceb41dd980a6ddb72546804e"}
Jul 29 21:46:02.096757 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.096584    2569 scope.go:117] "RemoveContainer" containerID="92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"
Jul 29 21:46:02.098319 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.098175    2569 generic.go:356] "Generic (PLEG): container finished" podID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" containerID="3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0" exitCode=0
Jul 29 21:46:02.098319 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.098238    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"
Jul 29 21:46:02.098319 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.098273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" event={"ID":"f05d4497-3b3f-4dd1-87f7-f36dccdb6768","Type":"ContainerDied","Data":"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"}
Jul 29 21:46:02.098484 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.098334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn" event={"ID":"f05d4497-3b3f-4dd1-87f7-f36dccdb6768","Type":"ContainerDied","Data":"1c200fa03891f4745a776d488b382e8210027380175594e7512f0d2135c4cb78"}
Jul 29 21:46:02.099857 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.099789    2569 generic.go:356] "Generic (PLEG): container finished" podID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerID="ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc" exitCode=0
Jul 29 21:46:02.099857 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.099823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" event={"ID":"a7eeb4bf-f46b-4701-b103-129148b74ad6","Type":"ContainerDied","Data":"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"}
Jul 29 21:46:02.099857 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.099844    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"
Jul 29 21:46:02.099857 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.099856    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4" event={"ID":"a7eeb4bf-f46b-4701-b103-129148b74ad6","Type":"ContainerDied","Data":"791cc5c191ef60e37e92b7b5666b40e73db949ce87214f05a128a3add1b99e49"}
Jul 29 21:46:02.108851 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.108836    2569 scope.go:117] "RemoveContainer" containerID="92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"
Jul 29 21:46:02.109088 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:46:02.109071    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d\": container with ID starting with 92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d not found: ID does not exist" containerID="92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"
Jul 29 21:46:02.109155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.109098    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d"} err="failed to get container status \"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d\": rpc error: code = NotFound desc = could not find container \"92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d\": container with ID starting with 92aaee322e863a121842ceb56efb82ae5c00fd884e058e9b7f60b1121a44790d not found: ID does not exist"
Jul 29 21:46:02.109155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.109121    2569 scope.go:117] "RemoveContainer" containerID="3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"
Jul 29 21:46:02.116085 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.116072    2569 scope.go:117] "RemoveContainer" containerID="3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"
Jul 29 21:46:02.116299 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:46:02.116283    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0\": container with ID starting with 3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0 not found: ID does not exist" containerID="3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"
Jul 29 21:46:02.116359 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.116304    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0"} err="failed to get container status \"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0\": rpc error: code = NotFound desc = could not find container \"3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0\": container with ID starting with 3517b565dc764dc88b029fac03690f337e0d6165dde8bbb945cc30bbbe19faf0 not found: ID does not exist"
Jul 29 21:46:02.116359 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.116318    2569 scope.go:117] "RemoveContainer" containerID="ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"
Jul 29 21:46:02.122764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.122750    2569 scope.go:117] "RemoveContainer" containerID="ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"
Jul 29 21:46:02.122993 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:46:02.122977    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc\": container with ID starting with ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc not found: ID does not exist" containerID="ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"
Jul 29 21:46:02.123043 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.122998    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc"} err="failed to get container status \"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc\": rpc error: code = NotFound desc = could not find container \"ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc\": container with ID starting with ee966f16aff041590911a9d572f70874e9609853e3801cdb16fefb9c8343b4bc not found: ID does not exist"
Jul 29 21:46:02.126211 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.126192    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:46:02.134307 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.134288    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-bdcc5a3e-7d79747f8d-8dqx4"]
Jul 29 21:46:02.145676 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.145654    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:46:02.155384 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.155363    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-bdcc5a3e-57d5ddcc94-xwgtn"]
Jul 29 21:46:02.168838 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.168817    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:46:02.173350 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.173327    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bdcc5a3e-bb79dd6d4-6cn92"]
Jul 29 21:46:02.458690 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.458621    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:02.567805 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:02.567783    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:03.897423 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:03.897389    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" path="/var/lib/kubelet/pods/a7eeb4bf-f46b-4701-b103-129148b74ad6/volumes"
Jul 29 21:46:03.897794 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:03.897726    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" path="/var/lib/kubelet/pods/f05d4497-3b3f-4dd1-87f7-f36dccdb6768/volumes"
Jul 29 21:46:03.898067 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:03.898054    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bb4a35-06ac-48c9-8c48-79f747232c85" path="/var/lib/kubelet/pods/f6bb4a35-06ac-48c9-8c48-79f747232c85/volumes"
Jul 29 21:46:03.936997 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:03.936971    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:04.037885 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:04.037852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:04.286037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:04.286013    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:04.591052 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:04.590986    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:04.772199 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:04.772164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:07.362647 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:07.362609    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:07.445629 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:07.445596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:08.563206 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563168    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563495    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563508    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563523    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bb4a35-06ac-48c9-8c48-79f747232c85" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563529    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bb4a35-06ac-48c9-8c48-79f747232c85" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563539    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" containerName="payload-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563544    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" containerName="payload-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563553    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerName="maas-api"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563558    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerName="maas-api"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563571    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" containerName="maas-api"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563576    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" containerName="maas-api"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563631    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bb4a35-06ac-48c9-8c48-79f747232c85" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563638    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05d4497-3b3f-4dd1-87f7-f36dccdb6768" containerName="payload-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563645    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2e50466-c763-4b4e-ba71-e2e3fcf53c4d" containerName="maas-api"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563651    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d5391e-3f4f-4afd-ac7a-2cbd37c8b656" containerName="payload-pre-processing"
Jul 29 21:46:08.563730 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.563658    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7eeb4bf-f46b-4701-b103-129148b74ad6" containerName="maas-api"
Jul 29 21:46:08.568325 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.568308    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.570722 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.570699    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b0cb4c-serving-cert\""
Jul 29 21:46:08.570864 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.570719    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:46:08.570864 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.570774    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:46:08.571562 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.571547    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:46:08.578257 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.578235    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:46:08.601796 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.601770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:08.710701 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.710678    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:08.726671 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.726647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.726805 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.726703    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cghmh\" (UniqueName: \"kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.726805 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.726781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.827645 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.827592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.827645 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.827633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cghmh\" (UniqueName: \"kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.827782 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.827667    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.828276 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.828261    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.829818 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.829801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.835373 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.835353    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cghmh\" (UniqueName: \"kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh\") pod \"maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:08.880002 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:08.879979    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:09.007426 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.007397    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:46:09.014257 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:46:09.014232    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c50eaad_af6a_46ee_8c79_29920581d220.slice/crio-fd0641438af5f2a9381949dd8ec611304cd4ac8117aaf9f606c98a2192704592 WatchSource:0}: Error finding container fd0641438af5f2a9381949dd8ec611304cd4ac8117aaf9f606c98a2192704592: Status 404 returned error can't find the container with id fd0641438af5f2a9381949dd8ec611304cd4ac8117aaf9f606c98a2192704592
Jul 29 21:46:09.015695 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.015677    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:46:09.134954 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.134888    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" event={"ID":"7c50eaad-af6a-46ee-8c79-29920581d220","Type":"ContainerStarted","Data":"fd0641438af5f2a9381949dd8ec611304cd4ac8117aaf9f606c98a2192704592"}
Jul 29 21:46:09.260696 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.260667    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:09.359480 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.359456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:09.366619 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.366596    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:46:09.371155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.371140    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.374821 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.374801    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b0cb4c\""
Jul 29 21:46:09.383146 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.383125    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:46:09.384762 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.384725    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:46:09.388287 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.388243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.391048 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.391027    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b0cb4c-dockercfg-2hkps\""
Jul 29 21:46:09.404752 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.404717    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:46:09.533725 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.533689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.533854 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.533799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9j9h\" (UniqueName: \"kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.533854 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.533840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mq8z\" (UniqueName: \"kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.533965 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.533897    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.634916 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.634889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mq8z\" (UniqueName: \"kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.635232 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.634934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.635232 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.634959    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.635232 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.634999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9j9h\" (UniqueName: \"kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.635587 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.635568    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.635625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.635612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.642707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.642659    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mq8z\" (UniqueName: \"kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z\") pod \"payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.642817 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.642796    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9j9h\" (UniqueName: \"kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h\") pod \"payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.681809 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.681790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:09.702398 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.702380    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:09.820687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.820658    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:46:09.824858 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:46:09.824815    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd8aa08_9ca8_4fe4_9fc2_cbcf93f0042d.slice/crio-13680931c60fb2e53fcc3d5dbcae2bfbda211478e766332d4955bd20c996eb37 WatchSource:0}: Error finding container 13680931c60fb2e53fcc3d5dbcae2bfbda211478e766332d4955bd20c996eb37: Status 404 returned error can't find the container with id 13680931c60fb2e53fcc3d5dbcae2bfbda211478e766332d4955bd20c996eb37
Jul 29 21:46:09.842138 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:09.842114    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:46:09.845520 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:46:09.845497    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e2576f_0a09_448b_a0cc_1b743bf9f3c6.slice/crio-ab680f3168ebe8fa27f21517427aef950144d9d8adda05ea5176430fd5c20d13 WatchSource:0}: Error finding container ab680f3168ebe8fa27f21517427aef950144d9d8adda05ea5176430fd5c20d13: Status 404 returned error can't find the container with id ab680f3168ebe8fa27f21517427aef950144d9d8adda05ea5176430fd5c20d13
Jul 29 21:46:10.141162 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.141131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" event={"ID":"18e2576f-0a09-448b-a0cc-1b743bf9f3c6","Type":"ContainerStarted","Data":"f40a07dad21efd19d41ed9b5ae6e5347faed0a7356d59d7213abfaf6935597d6"}
Jul 29 21:46:10.141350 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.141180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" event={"ID":"18e2576f-0a09-448b-a0cc-1b743bf9f3c6","Type":"ContainerStarted","Data":"ab680f3168ebe8fa27f21517427aef950144d9d8adda05ea5176430fd5c20d13"}
Jul 29 21:46:10.141350 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.141214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:10.142745 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.142709    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" event={"ID":"7c50eaad-af6a-46ee-8c79-29920581d220","Type":"ContainerStarted","Data":"ae5144aeae1111290806bd4cf4435d94d9ed472956f14e4c8fcfc7b9dad10227"}
Jul 29 21:46:10.142875 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.142779    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:10.144062 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.144038    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" event={"ID":"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d","Type":"ContainerStarted","Data":"f5678c3119ecf6ce8e90e1eab0de45b2ac1dc8f07acede3add776757d421fb17"}
Jul 29 21:46:10.144153 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.144071    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" event={"ID":"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d","Type":"ContainerStarted","Data":"13680931c60fb2e53fcc3d5dbcae2bfbda211478e766332d4955bd20c996eb37"}
Jul 29 21:46:10.144153 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.144091    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:10.157240 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.157200    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" podStartSLOduration=1.157188557 podStartE2EDuration="1.157188557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:46:10.156508643 +0000 UTC m=+3582.754485010" watchObservedRunningTime="2026-07-29 21:46:10.157188557 +0000 UTC m=+3582.755164913"
Jul 29 21:46:10.172414 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.172370    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" podStartSLOduration=1.691628592 podStartE2EDuration="2.172354437s" totalImagesPullingTime="480.725845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:46:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:46:10.172269674 +0000 UTC m=+3582.770246030" watchObservedRunningTime="2026-07-29 21:46:10.172354437 +0000 UTC m=+3582.770330793"
Jul 29 21:46:10.189569 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:10.189528    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" podStartSLOduration=1.189516052 podStartE2EDuration="1.189516052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:46:10.18785535 +0000 UTC m=+3582.785831711" watchObservedRunningTime="2026-07-29 21:46:10.189516052 +0000 UTC m=+3582.787492409"
Jul 29 21:46:12.954773 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:12.954723    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:46:16.154693 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:16.154663    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:46:21.150821 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:21.150746    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:46:21.151260 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:21.150828    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:46:32.388469 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:32.388435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:32.720366 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:32.720289    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:33.033479 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:33.033450    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:33.347124 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:33.347052    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:35.466428 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:35.466395    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:35.569569 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:35.569544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:36.558242 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.558205    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:46:36.563427 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.563403    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.565702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.565681    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ceb7ca-serving-cert\""
Jul 29 21:46:36.571096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.571040    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:46:36.597501 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.597472    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:36.643249 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.643226    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.643336 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.643261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.643336 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.643287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8g69\" (UniqueName: \"kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.744363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.744339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8g69\" (UniqueName: \"kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.744437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.744405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.744482 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.744440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.744554 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:46:36.744535    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-serving-cert: secret "maas-api-e2e-inf-b-ceb7ca-serving-cert" not found
Jul 29 21:46:36.744625 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:46:36.744613    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls podName:cf561e5c-7440-406a-ad81-6f0aac4a598f nodeName:}" failed. No retries permitted until 2026-07-29 21:46:37.244594424 +0000 UTC m=+3609.842570767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls") pod "maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" (UID: "cf561e5c-7440-406a-ad81-6f0aac4a598f") : secret "maas-api-e2e-inf-b-ceb7ca-serving-cert" not found
Jul 29 21:46:36.745015 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.745000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:36.753427 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:36.753402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8g69\" (UniqueName: \"kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:37.025486 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.025451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:37.126909 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.126868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:37.210466 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.210437    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:46:37.212602 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.212580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:37.213989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.213973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.216400 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.216366    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ceb7ca\""
Jul 29 21:46:37.216798 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.216779    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ceb7ca-dockercfg-cvjh6\""
Jul 29 21:46:37.222033 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.222010    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:46:37.248782 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.248756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:37.251197 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.251176    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") pod \"maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:37.349406 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.349343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.349509 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.349453    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppc8b\" (UniqueName: \"kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.450062 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.450037    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppc8b\" (UniqueName: \"kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.450193 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.450076    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.450672 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.450654    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.458886 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.458865    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppc8b\" (UniqueName: \"kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b\") pod \"payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.476678 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.476656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:37.525180 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.525127    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:37.621330 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.621157    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:46:37.625176 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:46:37.625144    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf561e5c_7440_406a_ad81_6f0aac4a598f.slice/crio-2d139c0509cb8b4a646f2776eb1ab5a58ec2cd9b90cb24fc1141b301b5bc3eb9 WatchSource:0}: Error finding container 2d139c0509cb8b4a646f2776eb1ab5a58ec2cd9b90cb24fc1141b301b5bc3eb9: Status 404 returned error can't find the container with id 2d139c0509cb8b4a646f2776eb1ab5a58ec2cd9b90cb24fc1141b301b5bc3eb9
Jul 29 21:46:37.666812 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:37.666789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:46:37.668521 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:46:37.668497    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09cade39_f5d3_48f3_8700_ffa68da460f0.slice/crio-fe6fe5b35b6c9ccc10a9c7d7ead25070a71d0064a8fc02705e667b3f3ba5a5a4 WatchSource:0}: Error finding container fe6fe5b35b6c9ccc10a9c7d7ead25070a71d0064a8fc02705e667b3f3ba5a5a4: Status 404 returned error can't find the container with id fe6fe5b35b6c9ccc10a9c7d7ead25070a71d0064a8fc02705e667b3f3ba5a5a4
Jul 29 21:46:38.267993 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.267911    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" event={"ID":"09cade39-f5d3-48f3-8700-ffa68da460f0","Type":"ContainerStarted","Data":"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"}
Jul 29 21:46:38.267993 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.267974    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" event={"ID":"09cade39-f5d3-48f3-8700-ffa68da460f0","Type":"ContainerStarted","Data":"fe6fe5b35b6c9ccc10a9c7d7ead25070a71d0064a8fc02705e667b3f3ba5a5a4"}
Jul 29 21:46:38.268189 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.268009    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:46:38.269151 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.269132    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" event={"ID":"cf561e5c-7440-406a-ad81-6f0aac4a598f","Type":"ContainerStarted","Data":"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"}
Jul 29 21:46:38.269243 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.269169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" event={"ID":"cf561e5c-7440-406a-ad81-6f0aac4a598f","Type":"ContainerStarted","Data":"2d139c0509cb8b4a646f2776eb1ab5a58ec2cd9b90cb24fc1141b301b5bc3eb9"}
Jul 29 21:46:38.269243 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.269219    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:38.285021 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.284986    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" podStartSLOduration=1.28497313 podStartE2EDuration="1.28497313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:46:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:46:38.284308402 +0000 UTC m=+3610.882284772" watchObservedRunningTime="2026-07-29 21:46:38.28497313 +0000 UTC m=+3610.882949485"
Jul 29 21:46:38.302832 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:38.302795    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" podStartSLOduration=1.936526298 podStartE2EDuration="2.302779369s" totalImagesPullingTime="366.253071ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:46:38.301033243 +0000 UTC m=+3610.899009611" watchObservedRunningTime="2026-07-29 21:46:38.302779369 +0000 UTC m=+3610.900755726"
Jul 29 21:46:43.052303 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:43.052269    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:46:44.284139 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:44.284109    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:46:45.480058 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:45.480024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:49.280930 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:46:49.280900    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:47:00.513580 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:00.513544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:47:06.373035 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:06.373002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:47:09.649484 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:09.649447    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:47:22.454728 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:22.454696    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:47:27.487981 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:27.487951    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:47:33.386154 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:33.386120    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:47:47.165524 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:47:47.165487    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:48:15.034705 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:15.034676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:16.393080 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:16.393043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:19.211513 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:19.211451    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:19.266784 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:19.266761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:22.854770 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:22.854723    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:48:26.553090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.553060    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:48:26.553534 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.553253    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" podUID="7c50eaad-af6a-46ee-8c79-29920581d220" containerName="maas-api" containerID="cri-o://ae5144aeae1111290806bd4cf4435d94d9ed472956f14e4c8fcfc7b9dad10227" gracePeriod=30
Jul 29 21:48:26.573389 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.573367    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:26.605467 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.605445    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:48:26.605654 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.605631    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" podUID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" containerName="payload-processing" containerID="cri-o://f40a07dad21efd19d41ed9b5ae6e5347faed0a7356d59d7213abfaf6935597d6" gracePeriod=30
Jul 29 21:48:26.612669 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.612376    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:48:26.612669 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.612573    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" podUID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" containerName="payload-pre-processing" containerID="cri-o://f5678c3119ecf6ce8e90e1eab0de45b2ac1dc8f07acede3add776757d421fb17" gracePeriod=30
Jul 29 21:48:26.644958 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.644932    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:26.703223 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.703200    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:26.770917 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.770788    2569 generic.go:356] "Generic (PLEG): container finished" podID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" containerID="f40a07dad21efd19d41ed9b5ae6e5347faed0a7356d59d7213abfaf6935597d6" exitCode=0
Jul 29 21:48:26.770917 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.770846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" event={"ID":"18e2576f-0a09-448b-a0cc-1b743bf9f3c6","Type":"ContainerDied","Data":"f40a07dad21efd19d41ed9b5ae6e5347faed0a7356d59d7213abfaf6935597d6"}
Jul 29 21:48:26.772593 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.772567    2569 generic.go:356] "Generic (PLEG): container finished" podID="7c50eaad-af6a-46ee-8c79-29920581d220" containerID="ae5144aeae1111290806bd4cf4435d94d9ed472956f14e4c8fcfc7b9dad10227" exitCode=0
Jul 29 21:48:26.772724 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.772646    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" event={"ID":"7c50eaad-af6a-46ee-8c79-29920581d220","Type":"ContainerDied","Data":"ae5144aeae1111290806bd4cf4435d94d9ed472956f14e4c8fcfc7b9dad10227"}
Jul 29 21:48:26.774318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.774284    2569 generic.go:356] "Generic (PLEG): container finished" podID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" containerID="f5678c3119ecf6ce8e90e1eab0de45b2ac1dc8f07acede3add776757d421fb17" exitCode=0
Jul 29 21:48:26.774420 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.774336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" event={"ID":"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d","Type":"ContainerDied","Data":"f5678c3119ecf6ce8e90e1eab0de45b2ac1dc8f07acede3add776757d421fb17"}
Jul 29 21:48:26.888671 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.888648    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:26.890813 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.890793    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:48:26.893524 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.893506    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:48:26.896390 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.896373    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:48:26.908006 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.907985    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9j9h\" (UniqueName: \"kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h\") pod \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") "
Jul 29 21:48:26.908101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908030    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls\") pod \"7c50eaad-af6a-46ee-8c79-29920581d220\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") "
Jul 29 21:48:26.908101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908062    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cghmh\" (UniqueName: \"kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh\") pod \"7c50eaad-af6a-46ee-8c79-29920581d220\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") "
Jul 29 21:48:26.908201 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908174    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca\") pod \"7c50eaad-af6a-46ee-8c79-29920581d220\" (UID: \"7c50eaad-af6a-46ee-8c79-29920581d220\") "
Jul 29 21:48:26.908256 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908236    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mq8z\" (UniqueName: \"kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z\") pod \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") "
Jul 29 21:48:26.908311 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908280    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume\") pod \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\" (UID: \"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d\") "
Jul 29 21:48:26.908422 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908326    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume\") pod \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\" (UID: \"18e2576f-0a09-448b-a0cc-1b743bf9f3c6\") "
Jul 29 21:48:26.908538 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908516    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c50eaad-af6a-46ee-8c79-29920581d220" (UID: "7c50eaad-af6a-46ee-8c79-29920581d220"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:48:26.908702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908682    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c50eaad-af6a-46ee-8c79-29920581d220-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:26.908951 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908913    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" (UID: "ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:48:26.909049 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.908948    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18e2576f-0a09-448b-a0cc-1b743bf9f3c6" (UID: "18e2576f-0a09-448b-a0cc-1b743bf9f3c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:48:26.910238 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.910209    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh" (OuterVolumeSpecName: "kube-api-access-cghmh") pod "7c50eaad-af6a-46ee-8c79-29920581d220" (UID: "7c50eaad-af6a-46ee-8c79-29920581d220"). InnerVolumeSpecName "kube-api-access-cghmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:48:26.910346 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.910322    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z" (OuterVolumeSpecName: "kube-api-access-8mq8z") pod "ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" (UID: "ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d"). InnerVolumeSpecName "kube-api-access-8mq8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:48:26.910505 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.910484    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h" (OuterVolumeSpecName: "kube-api-access-l9j9h") pod "18e2576f-0a09-448b-a0cc-1b743bf9f3c6" (UID: "18e2576f-0a09-448b-a0cc-1b743bf9f3c6"). InnerVolumeSpecName "kube-api-access-l9j9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:48:26.910505 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:26.910490    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c50eaad-af6a-46ee-8c79-29920581d220" (UID: "7c50eaad-af6a-46ee-8c79-29920581d220"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:48:27.009155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009126    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9j9h\" (UniqueName: \"kubernetes.io/projected/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-kube-api-access-l9j9h\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.009155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009150    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c50eaad-af6a-46ee-8c79-29920581d220-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.009155 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009159    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cghmh\" (UniqueName: \"kubernetes.io/projected/7c50eaad-af6a-46ee-8c79-29920581d220-kube-api-access-cghmh\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.009362 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009169    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mq8z\" (UniqueName: \"kubernetes.io/projected/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-kube-api-access-8mq8z\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.009362 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009177    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.009362 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.009187    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18e2576f-0a09-448b-a0cc-1b743bf9f3c6-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:27.206879 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.206806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:27.780630 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.780603    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"
Jul 29 21:48:27.781090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.780652    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7" event={"ID":"18e2576f-0a09-448b-a0cc-1b743bf9f3c6","Type":"ContainerDied","Data":"ab680f3168ebe8fa27f21517427aef950144d9d8adda05ea5176430fd5c20d13"}
Jul 29 21:48:27.781090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.780709    2569 scope.go:117] "RemoveContainer" containerID="f40a07dad21efd19d41ed9b5ae6e5347faed0a7356d59d7213abfaf6935597d6"
Jul 29 21:48:27.781903 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.781882    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"
Jul 29 21:48:27.781982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.781897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf" event={"ID":"7c50eaad-af6a-46ee-8c79-29920581d220","Type":"ContainerDied","Data":"fd0641438af5f2a9381949dd8ec611304cd4ac8117aaf9f606c98a2192704592"}
Jul 29 21:48:27.783332 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.783309    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm" event={"ID":"ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d","Type":"ContainerDied","Data":"13680931c60fb2e53fcc3d5dbcae2bfbda211478e766332d4955bd20c996eb37"}
Jul 29 21:48:27.783431 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.783332    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"
Jul 29 21:48:27.797209 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.797181    2569 scope.go:117] "RemoveContainer" containerID="ae5144aeae1111290806bd4cf4435d94d9ed472956f14e4c8fcfc7b9dad10227"
Jul 29 21:48:27.805714 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.805699    2569 scope.go:117] "RemoveContainer" containerID="f5678c3119ecf6ce8e90e1eab0de45b2ac1dc8f07acede3add776757d421fb17"
Jul 29 21:48:27.808764 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.808729    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:48:27.812327 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.812304    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b0cb4c-699d9f989d-kkgt7"]
Jul 29 21:48:27.822522 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.822469    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:48:27.826071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.826053    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b0cb4c-64dfb9b9dd-ffrvf"]
Jul 29 21:48:27.835338 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.835320    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:48:27.839611 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.839593    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b0cb4c-64647dcdc6-299hm"]
Jul 29 21:48:27.898014 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.897991    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" path="/var/lib/kubelet/pods/18e2576f-0a09-448b-a0cc-1b743bf9f3c6/volumes"
Jul 29 21:48:27.898387 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.898372    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c50eaad-af6a-46ee-8c79-29920581d220" path="/var/lib/kubelet/pods/7c50eaad-af6a-46ee-8c79-29920581d220/volumes"
Jul 29 21:48:27.898689 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:27.898676    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" path="/var/lib/kubelet/pods/ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d/volumes"
Jul 29 21:48:31.860496 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:31.860464    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:48:32.983065 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:32.983034    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:33.083234 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:33.083201    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:33.281980 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:33.281907    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:33.522904 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:33.522880    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.111811 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.111779    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:48:44.112256 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.112014    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerName="maas-api" containerID="cri-o://e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525" gracePeriod=30
Jul 29 21:48:44.133598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.133573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.175944 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.175915    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:48:44.176192 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.176164    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" podUID="09cade39-f5d3-48f3-8700-ffa68da460f0" containerName="payload-pre-processing" containerID="cri-o://25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a" gracePeriod=30
Jul 29 21:48:44.236122 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.236096    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.278687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.278662    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.54:8443/health\": dial tcp 10.134.0.54:8443: connect: connection refused"
Jul 29 21:48:44.313041 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.313011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.367988 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.367924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.462931 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.462906    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:48:44.532515 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.532489    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume\") pod \"09cade39-f5d3-48f3-8700-ffa68da460f0\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") "
Jul 29 21:48:44.532658 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.532569    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppc8b\" (UniqueName: \"kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b\") pod \"09cade39-f5d3-48f3-8700-ffa68da460f0\" (UID: \"09cade39-f5d3-48f3-8700-ffa68da460f0\") "
Jul 29 21:48:44.532859 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.532831    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09cade39-f5d3-48f3-8700-ffa68da460f0" (UID: "09cade39-f5d3-48f3-8700-ffa68da460f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:48:44.534828 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.534806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b" (OuterVolumeSpecName: "kube-api-access-ppc8b") pod "09cade39-f5d3-48f3-8700-ffa68da460f0" (UID: "09cade39-f5d3-48f3-8700-ffa68da460f0"). InnerVolumeSpecName "kube-api-access-ppc8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:48:44.559980 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.559963    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:48:44.633156 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633107    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8g69\" (UniqueName: \"kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69\") pod \"cf561e5c-7440-406a-ad81-6f0aac4a598f\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") "
Jul 29 21:48:44.633156 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") pod \"cf561e5c-7440-406a-ad81-6f0aac4a598f\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") "
Jul 29 21:48:44.633282 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633175    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca\") pod \"cf561e5c-7440-406a-ad81-6f0aac4a598f\" (UID: \"cf561e5c-7440-406a-ad81-6f0aac4a598f\") "
Jul 29 21:48:44.633369 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633352    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09cade39-f5d3-48f3-8700-ffa68da460f0-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:44.633413 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633375    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppc8b\" (UniqueName: \"kubernetes.io/projected/09cade39-f5d3-48f3-8700-ffa68da460f0-kube-api-access-ppc8b\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:44.633558 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.633537    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf561e5c-7440-406a-ad81-6f0aac4a598f" (UID: "cf561e5c-7440-406a-ad81-6f0aac4a598f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:48:44.635114 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.635091    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69" (OuterVolumeSpecName: "kube-api-access-v8g69") pod "cf561e5c-7440-406a-ad81-6f0aac4a598f" (UID: "cf561e5c-7440-406a-ad81-6f0aac4a598f"). InnerVolumeSpecName "kube-api-access-v8g69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:48:44.635114 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.635099    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf561e5c-7440-406a-ad81-6f0aac4a598f" (UID: "cf561e5c-7440-406a-ad81-6f0aac4a598f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:48:44.733777 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.733752    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8g69\" (UniqueName: \"kubernetes.io/projected/cf561e5c-7440-406a-ad81-6f0aac4a598f-kube-api-access-v8g69\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:44.733777 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.733773    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf561e5c-7440-406a-ad81-6f0aac4a598f-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:44.733917 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.733782    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf561e5c-7440-406a-ad81-6f0aac4a598f-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:48:44.734901 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.734882    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:44.862480 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.862450    2569 generic.go:356] "Generic (PLEG): container finished" podID="09cade39-f5d3-48f3-8700-ffa68da460f0" containerID="25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a" exitCode=0
Jul 29 21:48:44.862580 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.862515    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"
Jul 29 21:48:44.862580 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.862541    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" event={"ID":"09cade39-f5d3-48f3-8700-ffa68da460f0","Type":"ContainerDied","Data":"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"}
Jul 29 21:48:44.862699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.862629    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g" event={"ID":"09cade39-f5d3-48f3-8700-ffa68da460f0","Type":"ContainerDied","Data":"fe6fe5b35b6c9ccc10a9c7d7ead25070a71d0064a8fc02705e667b3f3ba5a5a4"}
Jul 29 21:48:44.862699 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.862661    2569 scope.go:117] "RemoveContainer" containerID="25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"
Jul 29 21:48:44.864082 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.864024    2569 generic.go:356] "Generic (PLEG): container finished" podID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerID="e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525" exitCode=0
Jul 29 21:48:44.864177 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.864087    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" event={"ID":"cf561e5c-7440-406a-ad81-6f0aac4a598f","Type":"ContainerDied","Data":"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"}
Jul 29 21:48:44.864177 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.864134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9" event={"ID":"cf561e5c-7440-406a-ad81-6f0aac4a598f","Type":"ContainerDied","Data":"2d139c0509cb8b4a646f2776eb1ab5a58ec2cd9b90cb24fc1141b301b5bc3eb9"}
Jul 29 21:48:44.864177 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.864065    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"
Jul 29 21:48:44.870606 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.870589    2569 scope.go:117] "RemoveContainer" containerID="25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"
Jul 29 21:48:44.870889 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:48:44.870869    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a\": container with ID starting with 25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a not found: ID does not exist" containerID="25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"
Jul 29 21:48:44.870972 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.870899    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a"} err="failed to get container status \"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a\": rpc error: code = NotFound desc = could not find container \"25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a\": container with ID starting with 25c987dc8175962da9bb8a312c750c3d5ec874f8ddd1cea0eb7a83ef19dc218a not found: ID does not exist"
Jul 29 21:48:44.870972 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.870920    2569 scope.go:117] "RemoveContainer" containerID="e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"
Jul 29 21:48:44.878365 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.878347    2569 scope.go:117] "RemoveContainer" containerID="e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"
Jul 29 21:48:44.878586 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:48:44.878568    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525\": container with ID starting with e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525 not found: ID does not exist" containerID="e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"
Jul 29 21:48:44.878631 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.878591    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525"} err="failed to get container status \"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525\": rpc error: code = NotFound desc = could not find container \"e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525\": container with ID starting with e3b2c8e0c11d9bd832f18df354add66ca1923a7f3da39e461f14cca323c09525 not found: ID does not exist"
Jul 29 21:48:44.887656 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.887609    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:48:44.894233 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.894214    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ceb7ca-7c8f4d494d-dwc7g"]
Jul 29 21:48:44.905254 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.905231    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:48:44.909015 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:44.908996    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ceb7ca-cfbd8c57d-cldq9"]
Jul 29 21:48:45.896528 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:45.896501    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09cade39-f5d3-48f3-8700-ffa68da460f0" path="/var/lib/kubelet/pods/09cade39-f5d3-48f3-8700-ffa68da460f0/volumes"
Jul 29 21:48:45.896888 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:45.896873    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" path="/var/lib/kubelet/pods/cf561e5c-7440-406a-ad81-6f0aac4a598f/volumes"
Jul 29 21:48:47.098167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:47.098130    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:48:50.460808 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:50.460770    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:50.565505 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:50.565467    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:50.774898 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:50.774865    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:51.002982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:51.002954    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:51.355470 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:51.355445    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:51.698567 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:51.698497    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:51.904935 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:51.904908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:54.139506 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:54.139476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:54.240634 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:54.240606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:55.254799 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.254768    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255125    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" containerName="payload-processing"
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255139    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" containerName="payload-processing"
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255148    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c50eaad-af6a-46ee-8c79-29920581d220" containerName="maas-api"
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255154    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c50eaad-af6a-46ee-8c79-29920581d220" containerName="maas-api"
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255162    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09cade39-f5d3-48f3-8700-ffa68da460f0" containerName="payload-pre-processing"
Jul 29 21:48:55.255167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255168    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09cade39-f5d3-48f3-8700-ffa68da460f0" containerName="payload-pre-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255176    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" containerName="payload-pre-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255182    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" containerName="payload-pre-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255190    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerName="maas-api"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255195    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerName="maas-api"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255275    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddd8aa08-9ca8-4fe4-9fc2-cbcf93f0042d" containerName="payload-pre-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255289    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="09cade39-f5d3-48f3-8700-ffa68da460f0" containerName="payload-pre-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255298    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf561e5c-7440-406a-ad81-6f0aac4a598f" containerName="maas-api"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255312    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e2576f-0a09-448b-a0cc-1b743bf9f3c6" containerName="payload-processing"
Jul 29 21:48:55.255356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.255324    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c50eaad-af6a-46ee-8c79-29920581d220" containerName="maas-api"
Jul 29 21:48:55.259812 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.259795    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.262423 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.262401    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:48:55.262543 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.262401    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fdb37a-serving-cert\""
Jul 29 21:48:55.262543 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.262401    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:48:55.262543 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.262404    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:48:55.268495 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.268472    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:48:55.305114 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.305091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz42w\" (UniqueName: \"kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.305246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.305161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.305246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.305208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.351129 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.351101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:55.405545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.405521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.405655 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.405563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.405655 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.405607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz42w\" (UniqueName: \"kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.405779 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:48:55.405731    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-serving-cert: secret "maas-api-e2e-api-a-fdb37a-serving-cert" not found
Jul 29 21:48:55.405836 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:48:55.405808    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls podName:939d2586-f24d-47a7-be45-9479add4af30 nodeName:}" failed. No retries permitted until 2026-07-29 21:48:55.905791488 +0000 UTC m=+3748.503767823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls") pod "maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" (UID: "939d2586-f24d-47a7-be45-9479add4af30") : secret "maas-api-e2e-api-a-fdb37a-serving-cert" not found
Jul 29 21:48:55.406199 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.406176    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.416085 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.416065    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz42w\" (UniqueName: \"kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.484931 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.484909    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:55.832644 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.832622    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:55.909274 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.909249    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.911782 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.911761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") pod \"maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:55.928036 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.928015    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:48:55.930357 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.930338    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:55.931275 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.931261    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:55.933650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.933632    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-fdb37a\""
Jul 29 21:48:55.941777 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.941756    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:48:55.968189 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.968164    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:48:55.971202 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.971189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:55.973509 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.973491    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-fdb37a-dockercfg-tsd5h\""
Jul 29 21:48:55.982160 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:55.982139    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:48:56.010132 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.010110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwj4m\" (UniqueName: \"kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.010239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.010139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.010239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.010165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27ns\" (UniqueName: \"kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.010239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.010214    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.111437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.111372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwj4m\" (UniqueName: \"kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.111437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.111407    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.111437 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.111432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27ns\" (UniqueName: \"kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.111619 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.111461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.112138 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.112121    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.112179 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.112157    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.120601 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.120582    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27ns\" (UniqueName: \"kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns\") pod \"payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.120773 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.120729    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwj4m\" (UniqueName: \"kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m\") pod \"payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") " pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.169650 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.169625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:56.241700 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.241668    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.279778 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.279731    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.302588 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:48:56.302553    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939d2586_f24d_47a7_be45_9479add4af30.slice/crio-169d6736bc895713f0f3dafdc0815773d4996ec2c33fa48e7ec9fbc96b9d650f WatchSource:0}: Error finding container 169d6736bc895713f0f3dafdc0815773d4996ec2c33fa48e7ec9fbc96b9d650f: Status 404 returned error can't find the container with id 169d6736bc895713f0f3dafdc0815773d4996ec2c33fa48e7ec9fbc96b9d650f
Jul 29 21:48:56.302705 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.302668    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:48:56.415887 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.415859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:48:56.417832 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:48:56.417797    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513e1e15_bdc1_44ff_9ded_4e7930b1f658.slice/crio-e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8 WatchSource:0}: Error finding container e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8: Status 404 returned error can't find the container with id e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8
Jul 29 21:48:56.589921 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.589895    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:48:56.592127 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:48:56.592105    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612c4b35_81c5_4471_8863_cbdc03895e18.slice/crio-d52c8ffd0e64dea282c5bddec1ab69697cb1ef85b79cbccf92ea33b1292560c0 WatchSource:0}: Error finding container d52c8ffd0e64dea282c5bddec1ab69697cb1ef85b79cbccf92ea33b1292560c0: Status 404 returned error can't find the container with id d52c8ffd0e64dea282c5bddec1ab69697cb1ef85b79cbccf92ea33b1292560c0
Jul 29 21:48:56.920183 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.920115    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" event={"ID":"513e1e15-bdc1-44ff-9ded-4e7930b1f658","Type":"ContainerStarted","Data":"8cedb9389ca555c133b7e874ee1f3561ec961d7fc56827b5be625af9b099a097"}
Jul 29 21:48:56.920183 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.920167    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" event={"ID":"513e1e15-bdc1-44ff-9ded-4e7930b1f658","Type":"ContainerStarted","Data":"e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8"}
Jul 29 21:48:56.920371 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.920221    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:48:56.921381 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.921360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" event={"ID":"939d2586-f24d-47a7-be45-9479add4af30","Type":"ContainerStarted","Data":"169d6736bc895713f0f3dafdc0815773d4996ec2c33fa48e7ec9fbc96b9d650f"}
Jul 29 21:48:56.922811 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.922791    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" event={"ID":"612c4b35-81c5-4471-8863-cbdc03895e18","Type":"ContainerStarted","Data":"c24a26d52b8179c23acb391bee597fc2ff38c49de129b2cd2b4f9ec8d9451f38"}
Jul 29 21:48:56.922913 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.922827    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" event={"ID":"612c4b35-81c5-4471-8863-cbdc03895e18","Type":"ContainerStarted","Data":"d52c8ffd0e64dea282c5bddec1ab69697cb1ef85b79cbccf92ea33b1292560c0"}
Jul 29 21:48:56.922963 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.922923    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:48:56.938549 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.938507    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" podStartSLOduration=1.93849311 podStartE2EDuration="1.93849311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:48:56.936621042 +0000 UTC m=+3749.534597418" watchObservedRunningTime="2026-07-29 21:48:56.93849311 +0000 UTC m=+3749.536469444"
Jul 29 21:48:56.955467 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:56.955425    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" podStartSLOduration=1.9554135719999999 podStartE2EDuration="1.955413572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:48:56.954388781 +0000 UTC m=+3749.552365155" watchObservedRunningTime="2026-07-29 21:48:56.955413572 +0000 UTC m=+3749.553389928"
Jul 29 21:48:57.928536 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:57.928503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" event={"ID":"939d2586-f24d-47a7-be45-9479add4af30","Type":"ContainerStarted","Data":"784ba915cadab9cc6328cfd3f08fcc37043e4c2d4599c6a581f54f80cb9132b9"}
Jul 29 21:48:57.928900 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:57.928567    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:48:57.947479 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:57.947433    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" podStartSLOduration=2.307380443 podStartE2EDuration="2.947420156s" totalImagesPullingTime="640.039713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:48:57.945504456 +0000 UTC m=+3750.543480812" watchObservedRunningTime="2026-07-29 21:48:57.947420156 +0000 UTC m=+3750.545396511"
Jul 29 21:48:58.680960 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:58.680923    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:59.453090 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:48:59.453059    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:49:03.938887 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:03.938860    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:49:07.931158 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:07.931124    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:49:07.931513 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:07.931254    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:49:19.419418 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:19.419379    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:19.893332 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:19.893285    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:20.214707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:20.214602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:20.318599 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:20.318548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:23.401480 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:23.401438    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:23.501789 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:23.501765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:24.439694 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.439662    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:49:24.442899 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.442878    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.445075 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.445057    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-25008f-serving-cert\""
Jul 29 21:49:24.454125 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.454100    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:49:24.477547 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.477523    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:24.521239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.521216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.521334 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.521249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.521334 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.521274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcjqx\" (UniqueName: \"kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.622141 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.622108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.622239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.622150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.622239 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.622190    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcjqx\" (UniqueName: \"kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.622346 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:49:24.622263    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-serving-cert: secret "maas-api-e2e-api-b-25008f-serving-cert" not found
Jul 29 21:49:24.622346 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:49:24.622323    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls podName:b9463e54-8b65-40b1-9ba7-a600665a09cc nodeName:}" failed. No retries permitted until 2026-07-29 21:49:25.122306074 +0000 UTC m=+3777.720282409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls") pod "maas-api-e2e-api-b-25008f-77bb9569ff-px87z" (UID: "b9463e54-8b65-40b1-9ba7-a600665a09cc") : secret "maas-api-e2e-api-b-25008f-serving-cert" not found
Jul 29 21:49:24.622695 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.622677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.630135 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.630113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcjqx\" (UniqueName: \"kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:24.694304 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:24.694245    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:25.065716 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.065688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:25.126696 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.126671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:25.129006 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.128983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") pod \"maas-api-e2e-api-b-25008f-77bb9569ff-px87z\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:25.171536 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.171512    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:49:25.175154 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.175139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.177505 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.177488    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-25008f-dockercfg-tp7v6\""
Jul 29 21:49:25.177601 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.177587    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-25008f\""
Jul 29 21:49:25.184211 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.184190    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:49:25.186191 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.186163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:25.198172 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.198147    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:49:25.202016 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.201997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.220066 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.220048    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:49:25.227222 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.227199    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s829l\" (UniqueName: \"kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.227341 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.227323    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.328000 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.327943    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.328000 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.327984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s829l\" (UniqueName: \"kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.328126 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.328030    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.328126 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.328048    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwb86\" (UniqueName: \"kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.328625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.328608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.335254 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.335234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s829l\" (UniqueName: \"kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l\") pod \"payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.356680 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.356662    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:25.428938 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.428901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.429088 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.428949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwb86\" (UniqueName: \"kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.429804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.429770    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.437226 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.437199    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwb86\" (UniqueName: \"kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86\") pod \"payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") " pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.485010 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.484979    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:25.485970 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.485948    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:49:25.487710 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:49:25.487685    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9463e54_8b65_40b1_9ba7_a600665a09cc.slice/crio-4c744f34d84c8df4ce09fe6081207c49fe97e921550e4d957f7aff25ff46e151 WatchSource:0}: Error finding container 4c744f34d84c8df4ce09fe6081207c49fe97e921550e4d957f7aff25ff46e151: Status 404 returned error can't find the container with id 4c744f34d84c8df4ce09fe6081207c49fe97e921550e4d957f7aff25ff46e151
Jul 29 21:49:25.511268 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.511245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:25.639310 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.639284    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:49:25.641816 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:49:25.641781    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod758e17cf_ea60_4c16_b081_8be98512a9fa.slice/crio-17fca8a434cfa10698ba5ce0bef7d9f43eccd5687acb2a7a0967bafc9e93e353 WatchSource:0}: Error finding container 17fca8a434cfa10698ba5ce0bef7d9f43eccd5687acb2a7a0967bafc9e93e353: Status 404 returned error can't find the container with id 17fca8a434cfa10698ba5ce0bef7d9f43eccd5687acb2a7a0967bafc9e93e353
Jul 29 21:49:25.662833 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:25.662809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:49:25.664213 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:49:25.664188    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12facb97_ece1_4406_808c_6eb07945e2a1.slice/crio-12bdc1c78d728d5f31061e4ee97454768d27a9064f013d02074004097cb07f6b WatchSource:0}: Error finding container 12bdc1c78d728d5f31061e4ee97454768d27a9064f013d02074004097cb07f6b: Status 404 returned error can't find the container with id 12bdc1c78d728d5f31061e4ee97454768d27a9064f013d02074004097cb07f6b
Jul 29 21:49:26.051802 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.051770    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" event={"ID":"758e17cf-ea60-4c16-b081-8be98512a9fa","Type":"ContainerStarted","Data":"d1d52c6b1a56d7562cb01db57b86e5fe15daa500120b66330430881af269aa9a"}
Jul 29 21:49:26.051936 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.051828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" event={"ID":"758e17cf-ea60-4c16-b081-8be98512a9fa","Type":"ContainerStarted","Data":"17fca8a434cfa10698ba5ce0bef7d9f43eccd5687acb2a7a0967bafc9e93e353"}
Jul 29 21:49:26.051936 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.051856    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:26.053164 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.053142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" event={"ID":"12facb97-ece1-4406-808c-6eb07945e2a1","Type":"ContainerStarted","Data":"d83cc09297218595b6ac75bd7ab96f212c3ba8f54394173e1feb0ce3b050c3a1"}
Jul 29 21:49:26.053274 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.053182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" event={"ID":"12facb97-ece1-4406-808c-6eb07945e2a1","Type":"ContainerStarted","Data":"12bdc1c78d728d5f31061e4ee97454768d27a9064f013d02074004097cb07f6b"}
Jul 29 21:49:26.053274 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.053202    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:26.054588 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.054563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" event={"ID":"b9463e54-8b65-40b1-9ba7-a600665a09cc","Type":"ContainerStarted","Data":"407a0b80d7a94f9178780eab632a6f73fa35eb5fb406c2bf3772d3fe323e6c8a"}
Jul 29 21:49:26.054692 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.054614    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" event={"ID":"b9463e54-8b65-40b1-9ba7-a600665a09cc","Type":"ContainerStarted","Data":"4c744f34d84c8df4ce09fe6081207c49fe97e921550e4d957f7aff25ff46e151"}
Jul 29 21:49:26.054692 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.054642    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:26.074253 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.074200    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" podStartSLOduration=1.074179568 podStartE2EDuration="1.074179568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:26.072877408 +0000 UTC m=+3778.670853764" watchObservedRunningTime="2026-07-29 21:49:26.074179568 +0000 UTC m=+3778.672155922"
Jul 29 21:49:26.092722 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.092683    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" podStartSLOduration=1.670109155 podStartE2EDuration="2.092669768s" totalImagesPullingTime="422.560613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:26.091061315 +0000 UTC m=+3778.689037693" watchObservedRunningTime="2026-07-29 21:49:26.092669768 +0000 UTC m=+3778.690646123"
Jul 29 21:49:26.108697 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:26.108662    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" podStartSLOduration=1.108647172 podStartE2EDuration="1.108647172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:26.107580002 +0000 UTC m=+3778.705556357" watchObservedRunningTime="2026-07-29 21:49:26.108647172 +0000 UTC m=+3778.706623529"
Jul 29 21:49:30.451874 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:30.451839    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:49:32.063927 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:32.063894    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:49:37.061167 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:37.061130    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:49:37.061578 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:37.061287    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:49:48.370032 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:48.370001    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:51.682005 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:49:51.681975    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:06.934062 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.934029    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:50:06.934540 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.934273    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" podUID="939d2586-f24d-47a7-be45-9479add4af30" containerName="maas-api" containerID="cri-o://784ba915cadab9cc6328cfd3f08fcc37043e4c2d4599c6a581f54f80cb9132b9" gracePeriod=30
Jul 29 21:50:06.938562 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.938524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:06.964647 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.964617    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:50:06.964927 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.964884    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" podUID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" containerName="payload-processing" containerID="cri-o://8cedb9389ca555c133b7e874ee1f3561ec961d7fc56827b5be625af9b099a097" gracePeriod=30
Jul 29 21:50:06.981802 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.977785    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:50:06.981802 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:06.978002    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" podUID="612c4b35-81c5-4471-8863-cbdc03895e18" containerName="payload-pre-processing" containerID="cri-o://c24a26d52b8179c23acb391bee597fc2ff38c49de129b2cd2b4f9ec8d9451f38" gracePeriod=30
Jul 29 21:50:07.038839 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.038812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:07.116025 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.115995    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:07.239073 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.239030    2569 generic.go:356] "Generic (PLEG): container finished" podID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" containerID="8cedb9389ca555c133b7e874ee1f3561ec961d7fc56827b5be625af9b099a097" exitCode=0
Jul 29 21:50:07.239292 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.239267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" event={"ID":"513e1e15-bdc1-44ff-9ded-4e7930b1f658","Type":"ContainerDied","Data":"8cedb9389ca555c133b7e874ee1f3561ec961d7fc56827b5be625af9b099a097"}
Jul 29 21:50:07.239350 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.239330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42" event={"ID":"513e1e15-bdc1-44ff-9ded-4e7930b1f658","Type":"ContainerDied","Data":"e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8"}
Jul 29 21:50:07.239398 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.239356    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0a3dc9b2b5f1af65c2c4b30be75677aa0ff553c6a3e8cb9a49c6c87b096c5e8"
Jul 29 21:50:07.241329 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.241303    2569 generic.go:356] "Generic (PLEG): container finished" podID="939d2586-f24d-47a7-be45-9479add4af30" containerID="784ba915cadab9cc6328cfd3f08fcc37043e4c2d4599c6a581f54f80cb9132b9" exitCode=0
Jul 29 21:50:07.241510 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.241490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" event={"ID":"939d2586-f24d-47a7-be45-9479add4af30","Type":"ContainerDied","Data":"784ba915cadab9cc6328cfd3f08fcc37043e4c2d4599c6a581f54f80cb9132b9"}
Jul 29 21:50:07.243526 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.243507    2569 generic.go:356] "Generic (PLEG): container finished" podID="612c4b35-81c5-4471-8863-cbdc03895e18" containerID="c24a26d52b8179c23acb391bee597fc2ff38c49de129b2cd2b4f9ec8d9451f38" exitCode=0
Jul 29 21:50:07.243617 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.243579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" event={"ID":"612c4b35-81c5-4471-8863-cbdc03895e18","Type":"ContainerDied","Data":"c24a26d52b8179c23acb391bee597fc2ff38c49de129b2cd2b4f9ec8d9451f38"}
Jul 29 21:50:07.259989 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.259971    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:50:07.274921 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.274890    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:50:07.283065 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.283047    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:50:07.438596 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438524    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwj4m\" (UniqueName: \"kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m\") pod \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") "
Jul 29 21:50:07.438596 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438567    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca\") pod \"939d2586-f24d-47a7-be45-9479add4af30\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") "
Jul 29 21:50:07.438804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume\") pod \"612c4b35-81c5-4471-8863-cbdc03895e18\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") "
Jul 29 21:50:07.438804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438635    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") pod \"939d2586-f24d-47a7-be45-9479add4af30\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") "
Jul 29 21:50:07.438804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438651    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz42w\" (UniqueName: \"kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w\") pod \"939d2586-f24d-47a7-be45-9479add4af30\" (UID: \"939d2586-f24d-47a7-be45-9479add4af30\") "
Jul 29 21:50:07.438804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438665    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27ns\" (UniqueName: \"kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns\") pod \"612c4b35-81c5-4471-8863-cbdc03895e18\" (UID: \"612c4b35-81c5-4471-8863-cbdc03895e18\") "
Jul 29 21:50:07.438804 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.438682    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume\") pod \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\" (UID: \"513e1e15-bdc1-44ff-9ded-4e7930b1f658\") "
Jul 29 21:50:07.439166 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.439125    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "939d2586-f24d-47a7-be45-9479add4af30" (UID: "939d2586-f24d-47a7-be45-9479add4af30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:07.439290 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.439188    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "513e1e15-bdc1-44ff-9ded-4e7930b1f658" (UID: "513e1e15-bdc1-44ff-9ded-4e7930b1f658"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:07.439290 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.439201    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "612c4b35-81c5-4471-8863-cbdc03895e18" (UID: "612c4b35-81c5-4471-8863-cbdc03895e18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:07.440665 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.440644    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "939d2586-f24d-47a7-be45-9479add4af30" (UID: "939d2586-f24d-47a7-be45-9479add4af30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:50:07.440834 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.440809    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m" (OuterVolumeSpecName: "kube-api-access-bwj4m") pod "513e1e15-bdc1-44ff-9ded-4e7930b1f658" (UID: "513e1e15-bdc1-44ff-9ded-4e7930b1f658"). InnerVolumeSpecName "kube-api-access-bwj4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:07.440991 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.440975    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w" (OuterVolumeSpecName: "kube-api-access-tz42w") pod "939d2586-f24d-47a7-be45-9479add4af30" (UID: "939d2586-f24d-47a7-be45-9479add4af30"). InnerVolumeSpecName "kube-api-access-tz42w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:07.441096 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.441082    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns" (OuterVolumeSpecName: "kube-api-access-k27ns") pod "612c4b35-81c5-4471-8863-cbdc03895e18" (UID: "612c4b35-81c5-4471-8863-cbdc03895e18"). InnerVolumeSpecName "kube-api-access-k27ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:07.506769 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.506726    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:07.540318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540299    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/612c4b35-81c5-4471-8863-cbdc03895e18-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540317    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939d2586-f24d-47a7-be45-9479add4af30-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540327    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz42w\" (UniqueName: \"kubernetes.io/projected/939d2586-f24d-47a7-be45-9479add4af30-kube-api-access-tz42w\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540335    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27ns\" (UniqueName: \"kubernetes.io/projected/612c4b35-81c5-4471-8863-cbdc03895e18-kube-api-access-k27ns\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540343    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/513e1e15-bdc1-44ff-9ded-4e7930b1f658-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540351    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwj4m\" (UniqueName: \"kubernetes.io/projected/513e1e15-bdc1-44ff-9ded-4e7930b1f658-kube-api-access-bwj4m\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.540430 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.540358    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939d2586-f24d-47a7-be45-9479add4af30-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:07.607208 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:07.607186    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:08.248939 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.248916    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"
Jul 29 21:50:08.249327 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.248971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4" event={"ID":"612c4b35-81c5-4471-8863-cbdc03895e18","Type":"ContainerDied","Data":"d52c8ffd0e64dea282c5bddec1ab69697cb1ef85b79cbccf92ea33b1292560c0"}
Jul 29 21:50:08.249327 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.249023    2569 scope.go:117] "RemoveContainer" containerID="c24a26d52b8179c23acb391bee597fc2ff38c49de129b2cd2b4f9ec8d9451f38"
Jul 29 21:50:08.251246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.250779    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"
Jul 29 21:50:08.251246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.250789    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"
Jul 29 21:50:08.251246 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.250776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh" event={"ID":"939d2586-f24d-47a7-be45-9479add4af30","Type":"ContainerDied","Data":"169d6736bc895713f0f3dafdc0815773d4996ec2c33fa48e7ec9fbc96b9d650f"}
Jul 29 21:50:08.258005 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.257988    2569 scope.go:117] "RemoveContainer" containerID="784ba915cadab9cc6328cfd3f08fcc37043e4c2d4599c6a581f54f80cb9132b9"
Jul 29 21:50:08.273559 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.273541    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:50:08.275976 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.275956    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fdb37a-fdc757c86-fkhzh"]
Jul 29 21:50:08.286338 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.286314    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:50:08.289944 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.289925    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-fdb37a-54ff4b5466-6lt42"]
Jul 29 21:50:08.300063 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.300040    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:50:08.303867 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:08.303848    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-fdb37a-749b6cff76-jl8c4"]
Jul 29 21:50:09.897465 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:09.897431    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" path="/var/lib/kubelet/pods/513e1e15-bdc1-44ff-9ded-4e7930b1f658/volumes"
Jul 29 21:50:09.897850 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:09.897768    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="612c4b35-81c5-4471-8863-cbdc03895e18" path="/var/lib/kubelet/pods/612c4b35-81c5-4471-8863-cbdc03895e18/volumes"
Jul 29 21:50:09.898101 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:09.898088    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="939d2586-f24d-47a7-be45-9479add4af30" path="/var/lib/kubelet/pods/939d2586-f24d-47a7-be45-9479add4af30/volumes"
Jul 29 21:50:13.275624 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:13.275589    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:50:13.894723 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:13.894690    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:14.096172 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:14.096140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:14.294830 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:14.294802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:14.397477 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:14.397454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:30.084365 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.084335    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:50:30.084800 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.084519    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" podUID="b9463e54-8b65-40b1-9ba7-a600665a09cc" containerName="maas-api" containerID="cri-o://407a0b80d7a94f9178780eab632a6f73fa35eb5fb406c2bf3772d3fe323e6c8a" gracePeriod=30
Jul 29 21:50:30.108085 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.108061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:30.132620 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.132591    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:50:30.132849 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.132824    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" podUID="12facb97-ece1-4406-808c-6eb07945e2a1" containerName="payload-processing" containerID="cri-o://d83cc09297218595b6ac75bd7ab96f212c3ba8f54394173e1feb0ce3b050c3a1" gracePeriod=30
Jul 29 21:50:30.139961 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.139938    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:50:30.140193 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.140146    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" podUID="758e17cf-ea60-4c16-b081-8be98512a9fa" containerName="payload-pre-processing" containerID="cri-o://d1d52c6b1a56d7562cb01db57b86e5fe15daa500120b66330430881af269aa9a" gracePeriod=30
Jul 29 21:50:30.208390 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.208368    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:30.289131 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.289092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:30.351201 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.351122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:30.352327 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.352301    2569 generic.go:356] "Generic (PLEG): container finished" podID="12facb97-ece1-4406-808c-6eb07945e2a1" containerID="d83cc09297218595b6ac75bd7ab96f212c3ba8f54394173e1feb0ce3b050c3a1" exitCode=0
Jul 29 21:50:30.352457 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.352371    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" event={"ID":"12facb97-ece1-4406-808c-6eb07945e2a1","Type":"ContainerDied","Data":"d83cc09297218595b6ac75bd7ab96f212c3ba8f54394173e1feb0ce3b050c3a1"}
Jul 29 21:50:30.353878 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.353854    2569 generic.go:356] "Generic (PLEG): container finished" podID="b9463e54-8b65-40b1-9ba7-a600665a09cc" containerID="407a0b80d7a94f9178780eab632a6f73fa35eb5fb406c2bf3772d3fe323e6c8a" exitCode=0
Jul 29 21:50:30.353973 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.353921    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" event={"ID":"b9463e54-8b65-40b1-9ba7-a600665a09cc","Type":"ContainerDied","Data":"407a0b80d7a94f9178780eab632a6f73fa35eb5fb406c2bf3772d3fe323e6c8a"}
Jul 29 21:50:30.355642 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.355619    2569 generic.go:356] "Generic (PLEG): container finished" podID="758e17cf-ea60-4c16-b081-8be98512a9fa" containerID="d1d52c6b1a56d7562cb01db57b86e5fe15daa500120b66330430881af269aa9a" exitCode=0
Jul 29 21:50:30.355920 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.355683    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" event={"ID":"758e17cf-ea60-4c16-b081-8be98512a9fa","Type":"ContainerDied","Data":"d1d52c6b1a56d7562cb01db57b86e5fe15daa500120b66330430881af269aa9a"}
Jul 29 21:50:30.407980 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.407956    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:50:30.419019 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.418996    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:50:30.426022 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.426004    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:50:30.517928 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.517902    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume\") pod \"758e17cf-ea60-4c16-b081-8be98512a9fa\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") "
Jul 29 21:50:30.518051 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.517946    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") pod \"b9463e54-8b65-40b1-9ba7-a600665a09cc\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") "
Jul 29 21:50:30.518051 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.517973    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s829l\" (UniqueName: \"kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l\") pod \"758e17cf-ea60-4c16-b081-8be98512a9fa\" (UID: \"758e17cf-ea60-4c16-b081-8be98512a9fa\") "
Jul 29 21:50:30.518051 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.517995    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca\") pod \"b9463e54-8b65-40b1-9ba7-a600665a09cc\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") "
Jul 29 21:50:30.518051 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518049    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwb86\" (UniqueName: \"kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86\") pod \"12facb97-ece1-4406-808c-6eb07945e2a1\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") "
Jul 29 21:50:30.518241 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518082    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcjqx\" (UniqueName: \"kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx\") pod \"b9463e54-8b65-40b1-9ba7-a600665a09cc\" (UID: \"b9463e54-8b65-40b1-9ba7-a600665a09cc\") "
Jul 29 21:50:30.518241 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518103    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume\") pod \"12facb97-ece1-4406-808c-6eb07945e2a1\" (UID: \"12facb97-ece1-4406-808c-6eb07945e2a1\") "
Jul 29 21:50:30.518494 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518461    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9463e54-8b65-40b1-9ba7-a600665a09cc" (UID: "b9463e54-8b65-40b1-9ba7-a600665a09cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:30.518602 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518572    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12facb97-ece1-4406-808c-6eb07945e2a1" (UID: "12facb97-ece1-4406-808c-6eb07945e2a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:30.518717 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.518699    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "758e17cf-ea60-4c16-b081-8be98512a9fa" (UID: "758e17cf-ea60-4c16-b081-8be98512a9fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:30.520174 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.520154    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9463e54-8b65-40b1-9ba7-a600665a09cc" (UID: "b9463e54-8b65-40b1-9ba7-a600665a09cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:50:30.520244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.520166    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l" (OuterVolumeSpecName: "kube-api-access-s829l") pod "758e17cf-ea60-4c16-b081-8be98512a9fa" (UID: "758e17cf-ea60-4c16-b081-8be98512a9fa"). InnerVolumeSpecName "kube-api-access-s829l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:30.520471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.520444    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx" (OuterVolumeSpecName: "kube-api-access-qcjqx") pod "b9463e54-8b65-40b1-9ba7-a600665a09cc" (UID: "b9463e54-8b65-40b1-9ba7-a600665a09cc"). InnerVolumeSpecName "kube-api-access-qcjqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:30.520471 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.520455    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86" (OuterVolumeSpecName: "kube-api-access-hwb86") pod "12facb97-ece1-4406-808c-6eb07945e2a1" (UID: "12facb97-ece1-4406-808c-6eb07945e2a1"). InnerVolumeSpecName "kube-api-access-hwb86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:30.618598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618548    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwb86\" (UniqueName: \"kubernetes.io/projected/12facb97-ece1-4406-808c-6eb07945e2a1-kube-api-access-hwb86\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618569    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcjqx\" (UniqueName: \"kubernetes.io/projected/b9463e54-8b65-40b1-9ba7-a600665a09cc-kube-api-access-qcjqx\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618578    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12facb97-ece1-4406-808c-6eb07945e2a1-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618589    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/758e17cf-ea60-4c16-b081-8be98512a9fa-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618598    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9463e54-8b65-40b1-9ba7-a600665a09cc-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618810 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618606    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s829l\" (UniqueName: \"kubernetes.io/projected/758e17cf-ea60-4c16-b081-8be98512a9fa-kube-api-access-s829l\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.618810 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.618614    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9463e54-8b65-40b1-9ba7-a600665a09cc-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:50:30.665249 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:30.665228    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:31.365933 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.365846    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"
Jul 29 21:50:31.365933 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.365852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp" event={"ID":"12facb97-ece1-4406-808c-6eb07945e2a1","Type":"ContainerDied","Data":"12bdc1c78d728d5f31061e4ee97454768d27a9064f013d02074004097cb07f6b"}
Jul 29 21:50:31.366441 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.365993    2569 scope.go:117] "RemoveContainer" containerID="d83cc09297218595b6ac75bd7ab96f212c3ba8f54394173e1feb0ce3b050c3a1"
Jul 29 21:50:31.367498 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.367376    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"
Jul 29 21:50:31.367498 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.367393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z" event={"ID":"b9463e54-8b65-40b1-9ba7-a600665a09cc","Type":"ContainerDied","Data":"4c744f34d84c8df4ce09fe6081207c49fe97e921550e4d957f7aff25ff46e151"}
Jul 29 21:50:31.368995 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.368959    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw" event={"ID":"758e17cf-ea60-4c16-b081-8be98512a9fa","Type":"ContainerDied","Data":"17fca8a434cfa10698ba5ce0bef7d9f43eccd5687acb2a7a0967bafc9e93e353"}
Jul 29 21:50:31.368995 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.368986    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"
Jul 29 21:50:31.376424 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.376397    2569 scope.go:117] "RemoveContainer" containerID="407a0b80d7a94f9178780eab632a6f73fa35eb5fb406c2bf3772d3fe323e6c8a"
Jul 29 21:50:31.384452 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.384433    2569 scope.go:117] "RemoveContainer" containerID="d1d52c6b1a56d7562cb01db57b86e5fe15daa500120b66330430881af269aa9a"
Jul 29 21:50:31.391625 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.391609    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:50:31.396118 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.396100    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-25008f-66fd4bdd76-rtqcp"]
Jul 29 21:50:31.405992 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.405974    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:50:31.409459 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.409440    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-25008f-5d9984c488-xsjhw"]
Jul 29 21:50:31.420609 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.420588    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:50:31.424357 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.424332    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-25008f-77bb9569ff-px87z"]
Jul 29 21:50:31.902676 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.902648    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12facb97-ece1-4406-808c-6eb07945e2a1" path="/var/lib/kubelet/pods/12facb97-ece1-4406-808c-6eb07945e2a1/volumes"
Jul 29 21:50:31.903082 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.903061    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="758e17cf-ea60-4c16-b081-8be98512a9fa" path="/var/lib/kubelet/pods/758e17cf-ea60-4c16-b081-8be98512a9fa/volumes"
Jul 29 21:50:31.903461 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:31.903442    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9463e54-8b65-40b1-9ba7-a600665a09cc" path="/var/lib/kubelet/pods/b9463e54-8b65-40b1-9ba7-a600665a09cc/volumes"
Jul 29 21:50:34.747845 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:34.747814    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:50:36.526837 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:36.526807    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:36.625189 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:36.625166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:36.790352 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:36.790277    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:37.016783 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:37.016749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:37.375588 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:37.375544    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:37.679212 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:37.679129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:37.864476 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:37.864442    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.645436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:40.645400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.760836 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:40.760765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:41.736013 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.735981    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 21:50:41.736495 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736476    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="939d2586-f24d-47a7-be45-9479add4af30" containerName="maas-api"
Jul 29 21:50:41.736545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736500    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="939d2586-f24d-47a7-be45-9479add4af30" containerName="maas-api"
Jul 29 21:50:41.736545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736518    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" containerName="payload-processing"
Jul 29 21:50:41.736545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736529    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" containerName="payload-processing"
Jul 29 21:50:41.736545 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736541    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12facb97-ece1-4406-808c-6eb07945e2a1" containerName="payload-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736550    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12facb97-ece1-4406-808c-6eb07945e2a1" containerName="payload-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736565    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="758e17cf-ea60-4c16-b081-8be98512a9fa" containerName="payload-pre-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736574    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="758e17cf-ea60-4c16-b081-8be98512a9fa" containerName="payload-pre-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736599    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="612c4b35-81c5-4471-8863-cbdc03895e18" containerName="payload-pre-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736608    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="612c4b35-81c5-4471-8863-cbdc03895e18" containerName="payload-pre-processing"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736629    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9463e54-8b65-40b1-9ba7-a600665a09cc" containerName="maas-api"
Jul 29 21:50:41.736687 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736637    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9463e54-8b65-40b1-9ba7-a600665a09cc" containerName="maas-api"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736716    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9463e54-8b65-40b1-9ba7-a600665a09cc" containerName="maas-api"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736727    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12facb97-ece1-4406-808c-6eb07945e2a1" containerName="payload-processing"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736758    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="513e1e15-bdc1-44ff-9ded-4e7930b1f658" containerName="payload-processing"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736769    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="939d2586-f24d-47a7-be45-9479add4af30" containerName="maas-api"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736780    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="612c4b35-81c5-4471-8863-cbdc03895e18" containerName="payload-pre-processing"
Jul 29 21:50:41.736955 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.736788    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="758e17cf-ea60-4c16-b081-8be98512a9fa" containerName="payload-pre-processing"
Jul 29 21:50:41.742256 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.742232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.745071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.745047    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b69e52-serving-cert\""
Jul 29 21:50:41.745176 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.745077    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-944pn\""
Jul 29 21:50:41.745176 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.745051    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 21:50:41.745988 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.745973    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 21:50:41.749713 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.749690    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 21:50:41.769130 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.769095    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:41.807326 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.807300    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.807424 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.807356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.807424 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.807397    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9btcn\" (UniqueName: \"kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.907962 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.907941    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.908068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.907981    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.908068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.908033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9btcn\" (UniqueName: \"kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.908213 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:50:41.908192    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-serving-cert: secret "maas-api-e2e-shared-a-b69e52-serving-cert" not found
Jul 29 21:50:41.908272 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:50:41.908262    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls podName:67f73e49-73a0-4b39-89d7-333193dbd812 nodeName:}" failed. No retries permitted until 2026-07-29 21:50:42.408241227 +0000 UTC m=+3855.006217568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls") pod "maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" (UID: "67f73e49-73a0-4b39-89d7-333193dbd812") : secret "maas-api-e2e-shared-a-b69e52-serving-cert" not found
Jul 29 21:50:41.908714 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.908691    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:41.916684 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:41.916657    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9btcn\" (UniqueName: \"kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:42.293329 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.293287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:42.411714 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.411681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:42.414117 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.414094    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") pod \"maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:42.457146 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.457122    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:42.557786 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.557717    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:42.597608 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.597579    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 21:50:42.600982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.600960    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.603510 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.603492    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-b69e52-dockercfg-x568p\""
Jul 29 21:50:42.603608 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.603530    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-b69e52\""
Jul 29 21:50:42.611653 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.611632    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 21:50:42.639830 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.639804    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 21:50:42.642971 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.642956    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.652526 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.652505    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 21:50:42.655057 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.655037    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:42.715071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.714897    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.715071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.714938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4v9x\" (UniqueName: \"kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.715071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.714957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.715071 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.714979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79kwm\" (UniqueName: \"kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.791384 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.791354    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 21:50:42.793884 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:50:42.793853    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f73e49_73a0_4b39_89d7_333193dbd812.slice/crio-1881c7b2f2f4c010f00abe41d9e5399580445da00f6a2dd2cb9580a8875a6acf WatchSource:0}: Error finding container 1881c7b2f2f4c010f00abe41d9e5399580445da00f6a2dd2cb9580a8875a6acf: Status 404 returned error can't find the container with id 1881c7b2f2f4c010f00abe41d9e5399580445da00f6a2dd2cb9580a8875a6acf
Jul 29 21:50:42.816140 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.816140 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816120    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4v9x\" (UniqueName: \"kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.816140 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.816353 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816237    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79kwm\" (UniqueName: \"kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.816885 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816864    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.816885 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.816881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.823801 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.823780    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4v9x\" (UniqueName: \"kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x\") pod \"payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:42.823801 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.823795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79kwm\" (UniqueName: \"kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm\") pod \"payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.911286 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.911261    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:42.952389 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:42.952356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:43.052891 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.052859    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 21:50:43.054766 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:50:43.054719    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36433b9c_9ca9_41c6_b5ae_481d432022d4.slice/crio-7072f09e118950ff43f61050da9eb293e39128580b573c48906bb39521044f27 WatchSource:0}: Error finding container 7072f09e118950ff43f61050da9eb293e39128580b573c48906bb39521044f27: Status 404 returned error can't find the container with id 7072f09e118950ff43f61050da9eb293e39128580b573c48906bb39521044f27
Jul 29 21:50:43.098460 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.098439    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 21:50:43.100162 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:50:43.100134    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96de5f3_6329_4c23_a6fa_56e046ad7112.slice/crio-698cc78f5716595d2c7c599350e1ba1d082f28a16d384b84291e03a692532cbe WatchSource:0}: Error finding container 698cc78f5716595d2c7c599350e1ba1d082f28a16d384b84291e03a692532cbe: Status 404 returned error can't find the container with id 698cc78f5716595d2c7c599350e1ba1d082f28a16d384b84291e03a692532cbe
Jul 29 21:50:43.423257 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.423176    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" event={"ID":"67f73e49-73a0-4b39-89d7-333193dbd812","Type":"ContainerStarted","Data":"1881c7b2f2f4c010f00abe41d9e5399580445da00f6a2dd2cb9580a8875a6acf"}
Jul 29 21:50:43.424657 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.424633    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" event={"ID":"36433b9c-9ca9-41c6-b5ae-481d432022d4","Type":"ContainerStarted","Data":"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"}
Jul 29 21:50:43.424793 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.424674    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" event={"ID":"36433b9c-9ca9-41c6-b5ae-481d432022d4","Type":"ContainerStarted","Data":"7072f09e118950ff43f61050da9eb293e39128580b573c48906bb39521044f27"}
Jul 29 21:50:43.424793 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.424744    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:43.426282 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.426260    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" event={"ID":"e96de5f3-6329-4c23-a6fa-56e046ad7112","Type":"ContainerStarted","Data":"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"}
Jul 29 21:50:43.426392 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.426304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" event={"ID":"e96de5f3-6329-4c23-a6fa-56e046ad7112","Type":"ContainerStarted","Data":"698cc78f5716595d2c7c599350e1ba1d082f28a16d384b84291e03a692532cbe"}
Jul 29 21:50:43.426392 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.426332    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:50:43.440602 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.440555    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" podStartSLOduration=1.440543214 podStartE2EDuration="1.440543214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:43.439668583 +0000 UTC m=+3856.037644961" watchObservedRunningTime="2026-07-29 21:50:43.440543214 +0000 UTC m=+3856.038519571"
Jul 29 21:50:43.455598 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:43.455559    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" podStartSLOduration=1.455548715 podStartE2EDuration="1.455548715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:43.454822909 +0000 UTC m=+3856.052799278" watchObservedRunningTime="2026-07-29 21:50:43.455548715 +0000 UTC m=+3856.053525072"
Jul 29 21:50:44.432443 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:44.432380    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" event={"ID":"67f73e49-73a0-4b39-89d7-333193dbd812","Type":"ContainerStarted","Data":"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"}
Jul 29 21:50:44.432967 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:44.432547    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:44.448003 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:44.447956    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" podStartSLOduration=2.795937201 podStartE2EDuration="3.447943478s" totalImagesPullingTime="652.006277ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:44.447025803 +0000 UTC m=+3857.045002184" watchObservedRunningTime="2026-07-29 21:50:44.447943478 +0000 UTC m=+3857.045919813"
Jul 29 21:50:44.956485 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:44.956449    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:50:50.443256 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:50.443180    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 21:50:54.434833 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:54.434797    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 21:50:54.435225 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:50:54.434873    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 21:51:05.667425 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:05.667380    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:06.073360 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:06.073330    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:06.334212 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:06.334129    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:06.559024 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:06.558987    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:08.797402 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:08.797369    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:08.898586 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:08.898541    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:10.048259 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.048217    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 21:51:10.052637 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.052614    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.054928 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.054906    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ac47f7-serving-cert\""
Jul 29 21:51:10.062645 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.062599    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 21:51:10.083426 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.083401    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:10.147797 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.147765    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.147938 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.147827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr9fp\" (UniqueName: \"kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.147938 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.147858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.248401 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.248360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr9fp\" (UniqueName: \"kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.248595 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.248419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.248595 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.248502    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.248595 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:51:10.248563    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-serving-cert: secret "maas-api-e2e-shared-b-ac47f7-serving-cert" not found
Jul 29 21:51:10.248779 ip-10-0-139-241 kubenswrapper[2569]: E0729 21:51:10.248632    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls podName:93920741-c808-44ba-abdb-e90990a66097 nodeName:}" failed. No retries permitted until 2026-07-29 21:51:10.748613558 +0000 UTC m=+3883.346589895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls") pod "maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" (UID: "93920741-c808-44ba-abdb-e90990a66097") : secret "maas-api-e2e-shared-b-ac47f7-serving-cert" not found
Jul 29 21:51:10.249107 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.249090    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.258982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.258955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr9fp\" (UniqueName: \"kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.311849 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.311748    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:10.537628 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.537585    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:10.637702 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.637629    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:10.680223 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.680193    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 21:51:10.683820 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.683799    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.686104 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.686083    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-ac47f7-dockercfg-bpbx9\""
Jul 29 21:51:10.686226 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.686124    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-ac47f7\""
Jul 29 21:51:10.693082 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.693061    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 21:51:10.752585 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.752557    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.752707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.752619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.752707 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.752639    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h79fm\" (UniqueName: \"kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.755068 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.755045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") pod \"maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.853436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.853409    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h79fm\" (UniqueName: \"kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.853670 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.853464    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.854244 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.854222    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.861233 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.861210    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h79fm\" (UniqueName: \"kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm\") pod \"payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:10.964336 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.964249    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:10.994340 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:10.994309    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:11.106674 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.106643    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 21:51:11.108687 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:51:11.108656    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93920741_c808_44ba_abdb_e90990a66097.slice/crio-a5954ae8157305ce2732ba94bf52ac099f688a18acca215878f6095ea05444e1 WatchSource:0}: Error finding container a5954ae8157305ce2732ba94bf52ac099f688a18acca215878f6095ea05444e1: Status 404 returned error can't find the container with id a5954ae8157305ce2732ba94bf52ac099f688a18acca215878f6095ea05444e1
Jul 29 21:51:11.110515 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.110494    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:51:11.140718 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.140699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 21:51:11.142962 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:51:11.142930    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80daedf_c89c_4db7_a6c6_fefeda62f594.slice/crio-43ca06dc4cfd5320b9dc53a70255a7fc3f251b9e7d1d71d21c16f60a920b7cc0 WatchSource:0}: Error finding container 43ca06dc4cfd5320b9dc53a70255a7fc3f251b9e7d1d71d21c16f60a920b7cc0: Status 404 returned error can't find the container with id 43ca06dc4cfd5320b9dc53a70255a7fc3f251b9e7d1d71d21c16f60a920b7cc0
Jul 29 21:51:11.566063 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.566032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" event={"ID":"93920741-c808-44ba-abdb-e90990a66097","Type":"ContainerStarted","Data":"a5954ae8157305ce2732ba94bf52ac099f688a18acca215878f6095ea05444e1"}
Jul 29 21:51:11.567594 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.567566    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" event={"ID":"e80daedf-c89c-4db7-a6c6-fefeda62f594","Type":"ContainerStarted","Data":"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"}
Jul 29 21:51:11.567698 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.567611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" event={"ID":"e80daedf-c89c-4db7-a6c6-fefeda62f594","Type":"ContainerStarted","Data":"43ca06dc4cfd5320b9dc53a70255a7fc3f251b9e7d1d71d21c16f60a920b7cc0"}
Jul 29 21:51:11.567698 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.567641    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:11.584195 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:11.584148    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" podStartSLOduration=1.584134563 podStartE2EDuration="1.584134563s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:51:11.583056964 +0000 UTC m=+3884.181033345" watchObservedRunningTime="2026-07-29 21:51:11.584134563 +0000 UTC m=+3884.182110898"
Jul 29 21:51:12.574280 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:12.574242    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" event={"ID":"93920741-c808-44ba-abdb-e90990a66097","Type":"ContainerStarted","Data":"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"}
Jul 29 21:51:12.574689 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:12.574444    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:12.590671 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:12.590610    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" podStartSLOduration=2.145251324 podStartE2EDuration="2.590592408s" totalImagesPullingTime="445.341084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:51:12.590059304 +0000 UTC m=+3885.188035663" watchObservedRunningTime="2026-07-29 21:51:12.590592408 +0000 UTC m=+3885.188568767"
Jul 29 21:51:16.253032 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:16.252993    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:51:18.587008 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:18.586976    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 21:51:22.576552 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:22.576519    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 21:51:33.827775 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:33.827722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:36.775767 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:36.775716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:39.964206 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:39.964167    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:51:54.373138 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:54.373087    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:58.355678 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:51:58.355640    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:52:34.855005 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:52:34.854913    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:52:59.535192 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:52:59.535161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:59.677081 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:52:59.677054    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:01.352630 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:53:01.352600    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:53:08.775077 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:53:08.775046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:19.854276 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:53:19.854239    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:53:47.374372 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:53:47.374339    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:54:05.289253 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:54:05.284206    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:54:32.762356 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:54:32.762324    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:54:50.959755 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:54:50.959706    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:55:15.205489 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:15.205455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:16.324367 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:16.324332    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:17.956607 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:17.956575    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:55:22.999979 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:22.999908    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:28.317523 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:28.317487    2569 scope.go:117] "RemoveContainer" containerID="8cedb9389ca555c133b7e874ee1f3561ec961d7fc56827b5be625af9b099a097"
Jul 29 21:55:35.851964 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:55:35.851930    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:56:03.254344 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:03.254306    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:56:17.562972 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:17.562936    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:56:26.354982 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:26.354947    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:56:51.458850 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:51.458760    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:56:56.716814 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:56.716766    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:56:57.523446 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:56:57.523412    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:08.471589 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:08.471546    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:57:14.372269 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:14.372233    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:17.263357 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:17.263324    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:57:33.647984 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.647952    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 21:57:33.651436 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.651416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.654281 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.654258    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ac47f7\"/\"openshift-service-ca.crt\""
Jul 29 21:57:33.654550 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.654534    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ac47f7\"/\"default-dockercfg-bcjjf\""
Jul 29 21:57:33.655500 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.655485    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ac47f7\"/\"kube-root-ca.crt\""
Jul 29 21:57:33.655620 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.655601    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ac47f7\"/\"sub-test-model-ac47f7-kserve-self-signed-certs\""
Jul 29 21:57:33.662245 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.662224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 21:57:33.679390 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.679368    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:33.759032 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.759160 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.759230 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh5rx\" (UniqueName: \"kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.759230 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.759318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.759318 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.759270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860597 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860565    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860681 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860681 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860681 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860848 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh5rx\" (UniqueName: \"kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860848 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.860975 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.860955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.861175 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.861158    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.861242 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.861178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.862902 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.862883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.863163 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.863137    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.869317 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.869293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh5rx\" (UniqueName: \"kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx\") pod \"sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") " pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:33.961363 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:33.961313    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:34.093087 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:34.093059    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 21:57:34.094764 ip-10-0-139-241 kubenswrapper[2569]: W0729 21:57:34.094716    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15104415_6c19_4f89_8b3c_2bb09c25af23.slice/crio-92ba479a6ad3ff0027fb8a1aafab9623c4cf195f12cf48841eeee13f953aac14 WatchSource:0}: Error finding container 92ba479a6ad3ff0027fb8a1aafab9623c4cf195f12cf48841eeee13f953aac14: Status 404 returned error can't find the container with id 92ba479a6ad3ff0027fb8a1aafab9623c4cf195f12cf48841eeee13f953aac14
Jul 29 21:57:34.298698 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:34.298669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" event={"ID":"15104415-6c19-4f89-8b3c-2bb09c25af23","Type":"ContainerStarted","Data":"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"}
Jul 29 21:57:34.298870 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:34.298708    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" event={"ID":"15104415-6c19-4f89-8b3c-2bb09c25af23","Type":"ContainerStarted","Data":"92ba479a6ad3ff0027fb8a1aafab9623c4cf195f12cf48841eeee13f953aac14"}
Jul 29 21:57:34.317779 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:34.317716    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" podStartSLOduration=1.317702326 podStartE2EDuration="1.317702326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:57:34.317613553 +0000 UTC m=+4266.915589906" watchObservedRunningTime="2026-07-29 21:57:34.317702326 +0000 UTC m=+4266.915678683"
Jul 29 21:57:37.448251 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:37.448213    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:57:43.961959 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:43.961923    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:43.962392 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:43.961974    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:43.974649 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:43.974619    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:57:44.356360 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:57:44.356328    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:58:31.359098 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:58:31.359020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:58:48.659609 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:58:48.659571    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:59:09.776516 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:09.776477    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:12.845744 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:12.845695    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:59:43.235087 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:43.235054    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:44.052631 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:44.052594    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 21:59:44.072786 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:44.072760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:49.756382 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:49.756345    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 21:59:55.969818 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:55.969782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:59.070928 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.070884    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" podUID="15104415-6c19-4f89-8b3c-2bb09c25af23" containerName="main" containerID="cri-o://d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3" gracePeriod=45
Jul 29 21:59:59.329211 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.329156    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:59:59.419899 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.419868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh5rx\" (UniqueName: \"kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.419935    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420037 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420157 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420050    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420157 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420081    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420157 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420110    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache\") pod \"15104415-6c19-4f89-8b3c-2bb09c25af23\" (UID: \"15104415-6c19-4f89-8b3c-2bb09c25af23\") "
Jul 29 21:59:59.420323 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420298    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home" (OuterVolumeSpecName: "home") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:59:59.420323 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420305    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:59:59.420426 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420382    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-home\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.420467 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.420410    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache" (OuterVolumeSpecName: "model-cache") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:59:59.422000 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.421975    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm" (OuterVolumeSpecName: "dshm") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:59:59.422110 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.422008    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx" (OuterVolumeSpecName: "kube-api-access-bh5rx") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "kube-api-access-bh5rx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:59:59.422110 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.422027    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15104415-6c19-4f89-8b3c-2bb09c25af23" (UID: "15104415-6c19-4f89-8b3c-2bb09c25af23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:59:59.520940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.520914    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-tmp-dir\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.520940 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.520938    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-dshm\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.521081 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.520947    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15104415-6c19-4f89-8b3c-2bb09c25af23-model-cache\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.521081 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.520956    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh5rx\" (UniqueName: \"kubernetes.io/projected/15104415-6c19-4f89-8b3c-2bb09c25af23-kube-api-access-bh5rx\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.521081 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.520965    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15104415-6c19-4f89-8b3c-2bb09c25af23-tls-certs\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 21:59:59.998670 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.998641    2569 generic.go:356] "Generic (PLEG): container finished" podID="15104415-6c19-4f89-8b3c-2bb09c25af23" containerID="d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3" exitCode=0
Jul 29 21:59:59.998809 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.998696    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"
Jul 29 21:59:59.998809 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.998697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" event={"ID":"15104415-6c19-4f89-8b3c-2bb09c25af23","Type":"ContainerDied","Data":"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"}
Jul 29 21:59:59.998896 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.998815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj" event={"ID":"15104415-6c19-4f89-8b3c-2bb09c25af23","Type":"ContainerDied","Data":"92ba479a6ad3ff0027fb8a1aafab9623c4cf195f12cf48841eeee13f953aac14"}
Jul 29 21:59:59.998896 ip-10-0-139-241 kubenswrapper[2569]: I0729 21:59:59.998841    2569 scope.go:117] "RemoveContainer" containerID="d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"
Jul 29 22:00:00.007407 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:00.007383    2569 scope.go:117] "RemoveContainer" containerID="d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"
Jul 29 22:00:00.007656 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:00:00.007633    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3\": container with ID starting with d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3 not found: ID does not exist" containerID="d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"
Jul 29 22:00:00.007714 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:00.007663    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3"} err="failed to get container status \"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3\": rpc error: code = NotFound desc = could not find container \"d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3\": container with ID starting with d73f87bfb6c858ce74372e54df2c8dcc087b2b8599b5c47c12091fe0868ef7a3 not found: ID does not exist"
Jul 29 22:00:00.014169 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:00.014145    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 22:00:00.019828 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:00.019809    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ac47f7/sub-test-model-ac47f7-kserve-7c6f5f9d99-nj8mj"]
Jul 29 22:00:00.058660 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:00.058638    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:00:01.898256 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:01.898224    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15104415-6c19-4f89-8b3c-2bb09c25af23" path="/var/lib/kubelet/pods/15104415-6c19-4f89-8b3c-2bb09c25af23/volumes"
Jul 29 22:00:20.518020 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:20.517982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:25.877463 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:25.877420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:29.451162 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:00:29.451129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:01:18.265998 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:01:18.265957    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:01:53.169964 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:01:53.169927    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:02:17.658389 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:17.658352    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:02:39.871116 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:39.871081    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:47.029194 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:47.029147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:47.968750 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:47.968696    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:53.847590 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:53.847507    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:02:58.375305 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.375262    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 22:02:58.375762 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.375532    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" podUID="67f73e49-73a0-4b39-89d7-333193dbd812" containerName="maas-api" containerID="cri-o://01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e" gracePeriod=30
Jul 29 22:02:58.430642 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.430607    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 22:02:58.430924 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.430867    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" podUID="36433b9c-9ca9-41c6-b5ae-481d432022d4" containerName="payload-processing" containerID="cri-o://5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20" gracePeriod=30
Jul 29 22:02:58.442792 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.442760    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 22:02:58.442980 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.442955    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" podUID="e96de5f3-6329-4c23-a6fa-56e046ad7112" containerName="payload-pre-processing" containerID="cri-o://2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad" gracePeriod=30
Jul 29 22:02:58.708459 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.708428    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 22:02:58.710444 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.710422    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 22:02:58.729301 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.729279    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 22:02:58.813403 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813376    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume\") pod \"e96de5f3-6329-4c23-a6fa-56e046ad7112\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") "
Jul 29 22:02:58.813403 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813404    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume\") pod \"36433b9c-9ca9-41c6-b5ae-481d432022d4\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") "
Jul 29 22:02:58.813650 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813463    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca\") pod \"67f73e49-73a0-4b39-89d7-333193dbd812\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") "
Jul 29 22:02:58.813650 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813489    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9btcn\" (UniqueName: \"kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn\") pod \"67f73e49-73a0-4b39-89d7-333193dbd812\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") "
Jul 29 22:02:58.813650 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813511    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4v9x\" (UniqueName: \"kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x\") pod \"e96de5f3-6329-4c23-a6fa-56e046ad7112\" (UID: \"e96de5f3-6329-4c23-a6fa-56e046ad7112\") "
Jul 29 22:02:58.813650 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813561    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") pod \"67f73e49-73a0-4b39-89d7-333193dbd812\" (UID: \"67f73e49-73a0-4b39-89d7-333193dbd812\") "
Jul 29 22:02:58.813650 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813617    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79kwm\" (UniqueName: \"kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm\") pod \"36433b9c-9ca9-41c6-b5ae-481d432022d4\" (UID: \"36433b9c-9ca9-41c6-b5ae-481d432022d4\") "
Jul 29 22:02:58.813956 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813861    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e96de5f3-6329-4c23-a6fa-56e046ad7112" (UID: "e96de5f3-6329-4c23-a6fa-56e046ad7112"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:58.813956 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813950    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e96de5f3-6329-4c23-a6fa-56e046ad7112-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.814064 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.813966    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36433b9c-9ca9-41c6-b5ae-481d432022d4" (UID: "36433b9c-9ca9-41c6-b5ae-481d432022d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:58.814228 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.814205    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67f73e49-73a0-4b39-89d7-333193dbd812" (UID: "67f73e49-73a0-4b39-89d7-333193dbd812"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:02:58.816150 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.816124    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn" (OuterVolumeSpecName: "kube-api-access-9btcn") pod "67f73e49-73a0-4b39-89d7-333193dbd812" (UID: "67f73e49-73a0-4b39-89d7-333193dbd812"). InnerVolumeSpecName "kube-api-access-9btcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:02:58.816279 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.816257    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x" (OuterVolumeSpecName: "kube-api-access-c4v9x") pod "e96de5f3-6329-4c23-a6fa-56e046ad7112" (UID: "e96de5f3-6329-4c23-a6fa-56e046ad7112"). InnerVolumeSpecName "kube-api-access-c4v9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:02:58.816358 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.816336    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67f73e49-73a0-4b39-89d7-333193dbd812" (UID: "67f73e49-73a0-4b39-89d7-333193dbd812"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:02:58.816429 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.816391    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm" (OuterVolumeSpecName: "kube-api-access-79kwm") pod "36433b9c-9ca9-41c6-b5ae-481d432022d4" (UID: "36433b9c-9ca9-41c6-b5ae-481d432022d4"). InnerVolumeSpecName "kube-api-access-79kwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:02:58.816982 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.816960    2569 generic.go:356] "Generic (PLEG): container finished" podID="e96de5f3-6329-4c23-a6fa-56e046ad7112" containerID="2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad" exitCode=0
Jul 29 22:02:58.817063 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.817008    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"
Jul 29 22:02:58.817063 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.817040    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" event={"ID":"e96de5f3-6329-4c23-a6fa-56e046ad7112","Type":"ContainerDied","Data":"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"}
Jul 29 22:02:58.817171 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.817122    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5" event={"ID":"e96de5f3-6329-4c23-a6fa-56e046ad7112","Type":"ContainerDied","Data":"698cc78f5716595d2c7c599350e1ba1d082f28a16d384b84291e03a692532cbe"}
Jul 29 22:02:58.817171 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.817153    2569 scope.go:117] "RemoveContainer" containerID="2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"
Jul 29 22:02:58.818374 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.818357    2569 generic.go:356] "Generic (PLEG): container finished" podID="67f73e49-73a0-4b39-89d7-333193dbd812" containerID="01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e" exitCode=0
Jul 29 22:02:58.818443 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.818395    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"
Jul 29 22:02:58.818510 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.818440    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" event={"ID":"67f73e49-73a0-4b39-89d7-333193dbd812","Type":"ContainerDied","Data":"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"}
Jul 29 22:02:58.818510 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.818490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp" event={"ID":"67f73e49-73a0-4b39-89d7-333193dbd812","Type":"ContainerDied","Data":"1881c7b2f2f4c010f00abe41d9e5399580445da00f6a2dd2cb9580a8875a6acf"}
Jul 29 22:02:58.819827 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.819802    2569 generic.go:356] "Generic (PLEG): container finished" podID="36433b9c-9ca9-41c6-b5ae-481d432022d4" containerID="5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20" exitCode=0
Jul 29 22:02:58.819920 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.819852    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"
Jul 29 22:02:58.819920 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.819883    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" event={"ID":"36433b9c-9ca9-41c6-b5ae-481d432022d4","Type":"ContainerDied","Data":"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"}
Jul 29 22:02:58.820007 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.819922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg" event={"ID":"36433b9c-9ca9-41c6-b5ae-481d432022d4","Type":"ContainerDied","Data":"7072f09e118950ff43f61050da9eb293e39128580b573c48906bb39521044f27"}
Jul 29 22:02:58.827128 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.827112    2569 scope.go:117] "RemoveContainer" containerID="2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"
Jul 29 22:02:58.827482 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:02:58.827453    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad\": container with ID starting with 2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad not found: ID does not exist" containerID="2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"
Jul 29 22:02:58.827565 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.827493    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad"} err="failed to get container status \"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad\": rpc error: code = NotFound desc = could not find container \"2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad\": container with ID starting with 2a26e459aa8cd67b613dfcdced72200d7184d74b94e17b702aaa612163dd3cad not found: ID does not exist"
Jul 29 22:02:58.827565 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.827517    2569 scope.go:117] "RemoveContainer" containerID="01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"
Jul 29 22:02:58.835820 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.835801    2569 scope.go:117] "RemoveContainer" containerID="01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"
Jul 29 22:02:58.836037 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:02:58.836020    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e\": container with ID starting with 01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e not found: ID does not exist" containerID="01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"
Jul 29 22:02:58.836077 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.836043    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e"} err="failed to get container status \"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e\": rpc error: code = NotFound desc = could not find container \"01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e\": container with ID starting with 01f5a774c3c126e5ca7dc2211171898a2680ccf27a2f6024bd53742b6024215e not found: ID does not exist"
Jul 29 22:02:58.836077 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.836059    2569 scope.go:117] "RemoveContainer" containerID="5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"
Jul 29 22:02:58.843572 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.843552    2569 scope.go:117] "RemoveContainer" containerID="5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"
Jul 29 22:02:58.843854 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:02:58.843831    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20\": container with ID starting with 5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20 not found: ID does not exist" containerID="5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"
Jul 29 22:02:58.843941 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.843862    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20"} err="failed to get container status \"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20\": rpc error: code = NotFound desc = could not find container \"5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20\": container with ID starting with 5b17bc0ab0f001553b280b8fdee9f028d37d7a6e0524191a5a99bc552718ba20 not found: ID does not exist"
Jul 29 22:02:58.843941 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.843922    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 22:02:58.846012 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.845990    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-b69e52-7d74dc7d56-6rtz5"]
Jul 29 22:02:58.856283 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.856261    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 22:02:58.861549 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.861531    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b69e52-85cfc97f7-mgpxp"]
Jul 29 22:02:58.870832 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.870811    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 22:02:58.875503 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.875483    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-b69e52-6457c9c8cd-5g2qg"]
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914501    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9btcn\" (UniqueName: \"kubernetes.io/projected/67f73e49-73a0-4b39-89d7-333193dbd812-kube-api-access-9btcn\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914523    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4v9x\" (UniqueName: \"kubernetes.io/projected/e96de5f3-6329-4c23-a6fa-56e046ad7112-kube-api-access-c4v9x\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914535    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67f73e49-73a0-4b39-89d7-333193dbd812-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79kwm\" (UniqueName: \"kubernetes.io/projected/36433b9c-9ca9-41c6-b5ae-481d432022d4-kube-api-access-79kwm\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914553    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36433b9c-9ca9-41c6-b5ae-481d432022d4-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:58.914558 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:58.914561    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67f73e49-73a0-4b39-89d7-333193dbd812-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:02:59.470712 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:59.470682    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:59.571706 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:59.571679    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:59.897411 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:59.897382    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36433b9c-9ca9-41c6-b5ae-481d432022d4" path="/var/lib/kubelet/pods/36433b9c-9ca9-41c6-b5ae-481d432022d4/volumes"
Jul 29 22:02:59.897783 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:59.897769    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67f73e49-73a0-4b39-89d7-333193dbd812" path="/var/lib/kubelet/pods/67f73e49-73a0-4b39-89d7-333193dbd812/volumes"
Jul 29 22:02:59.898091 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:02:59.898080    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e96de5f3-6329-4c23-a6fa-56e046ad7112" path="/var/lib/kubelet/pods/e96de5f3-6329-4c23-a6fa-56e046ad7112/volumes"
Jul 29 22:03:03.654879 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:03.654846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:03:04.796171 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:04.796140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:04.896963 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:04.896937    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:05.145798 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:05.145705    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:05.326051 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:05.326024    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:15.858004 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:15.857966    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 22:03:15.858604 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:15.858225    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" podUID="93920741-c808-44ba-abdb-e90990a66097" containerName="maas-api" containerID="cri-o://274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed" gracePeriod=30
Jul 29 22:03:15.918309 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:15.918277    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 22:03:15.918691 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:15.918664    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:15.918823 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:15.918781    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" podUID="e80daedf-c89c-4db7-a6c6-fefeda62f594" containerName="payload-processing" containerID="cri-o://bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c" gracePeriod=30
Jul 29 22:03:16.011028 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.010996    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:16.067455 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.067418    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:16.156524 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.156494    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 22:03:16.187353 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.187324    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 22:03:16.256970 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.256947    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") pod \"93920741-c808-44ba-abdb-e90990a66097\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") "
Jul 29 22:03:16.257108 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.257008    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca\") pod \"93920741-c808-44ba-abdb-e90990a66097\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") "
Jul 29 22:03:16.257108 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.257040    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vr9fp\" (UniqueName: \"kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp\") pod \"93920741-c808-44ba-abdb-e90990a66097\" (UID: \"93920741-c808-44ba-abdb-e90990a66097\") "
Jul 29 22:03:16.257393 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.257365    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93920741-c808-44ba-abdb-e90990a66097" (UID: "93920741-c808-44ba-abdb-e90990a66097"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:16.259036 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.259006    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp" (OuterVolumeSpecName: "kube-api-access-vr9fp") pod "93920741-c808-44ba-abdb-e90990a66097" (UID: "93920741-c808-44ba-abdb-e90990a66097"). InnerVolumeSpecName "kube-api-access-vr9fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:16.259111 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.259077    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93920741-c808-44ba-abdb-e90990a66097" (UID: "93920741-c808-44ba-abdb-e90990a66097"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:03:16.357917 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.357894    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume\") pod \"e80daedf-c89c-4db7-a6c6-fefeda62f594\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") "
Jul 29 22:03:16.358024 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.357954    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h79fm\" (UniqueName: \"kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm\") pod \"e80daedf-c89c-4db7-a6c6-fefeda62f594\" (UID: \"e80daedf-c89c-4db7-a6c6-fefeda62f594\") "
Jul 29 22:03:16.358206 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.358182    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93920741-c808-44ba-abdb-e90990a66097-openshift-service-ca\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:03:16.358254 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.358212    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr9fp\" (UniqueName: \"kubernetes.io/projected/93920741-c808-44ba-abdb-e90990a66097-kube-api-access-vr9fp\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:03:16.358254 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.358229    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93920741-c808-44ba-abdb-e90990a66097-maas-api-tls\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:03:16.358254 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.358210    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e80daedf-c89c-4db7-a6c6-fefeda62f594" (UID: "e80daedf-c89c-4db7-a6c6-fefeda62f594"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:16.359708 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.359689    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm" (OuterVolumeSpecName: "kube-api-access-h79fm") pod "e80daedf-c89c-4db7-a6c6-fefeda62f594" (UID: "e80daedf-c89c-4db7-a6c6-fefeda62f594"). InnerVolumeSpecName "kube-api-access-h79fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:16.440155 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.440097    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:16.459234 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.459213    2569 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80daedf-c89c-4db7-a6c6-fefeda62f594-plugins-config-volume\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:03:16.459234 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.459232    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h79fm\" (UniqueName: \"kubernetes.io/projected/e80daedf-c89c-4db7-a6c6-fefeda62f594-kube-api-access-h79fm\") on node \"ip-10-0-139-241.ec2.internal\" DevicePath \"\""
Jul 29 22:03:16.905160 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.905127    2569 generic.go:356] "Generic (PLEG): container finished" podID="93920741-c808-44ba-abdb-e90990a66097" containerID="274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed" exitCode=0
Jul 29 22:03:16.905549 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.905180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" event={"ID":"93920741-c808-44ba-abdb-e90990a66097","Type":"ContainerDied","Data":"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"}
Jul 29 22:03:16.905549 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.905234    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt" event={"ID":"93920741-c808-44ba-abdb-e90990a66097","Type":"ContainerDied","Data":"a5954ae8157305ce2732ba94bf52ac099f688a18acca215878f6095ea05444e1"}
Jul 29 22:03:16.905549 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.905188    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"
Jul 29 22:03:16.905549 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.905263    2569 scope.go:117] "RemoveContainer" containerID="274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"
Jul 29 22:03:16.906860 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.906584    2569 generic.go:356] "Generic (PLEG): container finished" podID="e80daedf-c89c-4db7-a6c6-fefeda62f594" containerID="bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c" exitCode=0
Jul 29 22:03:16.906860 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.906630    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" event={"ID":"e80daedf-c89c-4db7-a6c6-fefeda62f594","Type":"ContainerDied","Data":"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"}
Jul 29 22:03:16.906860 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.906640    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"
Jul 29 22:03:16.906860 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.906666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f" event={"ID":"e80daedf-c89c-4db7-a6c6-fefeda62f594","Type":"ContainerDied","Data":"43ca06dc4cfd5320b9dc53a70255a7fc3f251b9e7d1d71d21c16f60a920b7cc0"}
Jul 29 22:03:16.917535 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.917383    2569 scope.go:117] "RemoveContainer" containerID="274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"
Jul 29 22:03:16.917993 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:03:16.917967    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed\": container with ID starting with 274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed not found: ID does not exist" containerID="274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"
Jul 29 22:03:16.918112 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.918003    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed"} err="failed to get container status \"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed\": rpc error: code = NotFound desc = could not find container \"274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed\": container with ID starting with 274d297a7f1388de8aa1444c60e3fbc90efd64c91206d45cb09baed28f57ceed not found: ID does not exist"
Jul 29 22:03:16.918112 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.918026    2569 scope.go:117] "RemoveContainer" containerID="bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"
Jul 29 22:03:16.927215 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.927197    2569 scope.go:117] "RemoveContainer" containerID="bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"
Jul 29 22:03:16.927474 ip-10-0-139-241 kubenswrapper[2569]: E0729 22:03:16.927455    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c\": container with ID starting with bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c not found: ID does not exist" containerID="bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"
Jul 29 22:03:16.927532 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.927485    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c"} err="failed to get container status \"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c\": rpc error: code = NotFound desc = could not find container \"bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c\": container with ID starting with bda7fb9c3c6f0d1e0b07a983cb563aa00d9793c4bb2af48ec81e5a674680194c not found: ID does not exist"
Jul 29 22:03:16.932303 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.932280    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 22:03:16.934604 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.934584    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ac47f7-6c58c64f5c-hczxt"]
Jul 29 22:03:16.944604 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.944584    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 22:03:16.947519 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:16.947497    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ac47f7-796d945d5d-jdd6f"]
Jul 29 22:03:17.897948 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:17.897905    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93920741-c808-44ba-abdb-e90990a66097" path="/var/lib/kubelet/pods/93920741-c808-44ba-abdb-e90990a66097/volumes"
Jul 29 22:03:17.898369 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:17.898348    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e80daedf-c89c-4db7-a6c6-fefeda62f594" path="/var/lib/kubelet/pods/e80daedf-c89c-4db7-a6c6-fefeda62f594/volumes"
Jul 29 22:03:19.051049 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:19.051017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-7rcwp"]
Jul 29 22:03:22.283044 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:22.283006    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:22.341495 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:22.341461    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:22.560493 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:22.560416    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:22.801029 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:22.800995    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:29.254269 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:29.254240    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dbc94cc7-h954v_096a549e-855d-439d-a839-755ad9b7387d/copy-manifests/0.log"
Jul 29 22:03:29.260400 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:29.260373    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dbc94cc7-h954v_096a549e-855d-439d-a839-755ad9b7387d/manager/0.log"
Jul 29 22:03:29.355486 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:29.355456    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:31.835248 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:31.835219    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qjskm_5ff415ce-f7eb-4751-997a-b39a7ac75fad/kuadrant-console-plugin/0.log"
Jul 29 22:03:32.198706 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:32.198632    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7rcwp_8013d924-e01f-4ca5-9bde-2e5ed1fdc288/limitador/0.log"
Jul 29 22:03:32.889502 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:32.889474    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b585ddb49-whcx8_ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08/kube-auth-proxy/0.log"
Jul 29 22:03:33.908909 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:33.908878    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6xkck_815a0c7e-6733-45dc-86d6-de441fd4af96/main/0.log"
Jul 29 22:03:34.383101 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:34.383070    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-vjfl7_3e83f419-99fa-4d38-a9cf-01d3230a5cb8/main/0.log"
Jul 29 22:03:39.557278 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:39.557236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:39.662776 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:39.662747    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:43.067141 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:43.067113    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-m96cs_2a65e073-18d7-43d6-a600-eab03235eff9/global-pull-secret-syncer/0.log"
Jul 29 22:03:43.160936 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:43.160914    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c5v7z_ec015c0d-0ae9-4d65-97e5-6f54258963e1/konnectivity-agent/0.log"
Jul 29 22:03:43.319709 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:43.319646    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-241.ec2.internal_9347e8bed064fb1726f4031c3510ca65/haproxy/0.log"
Jul 29 22:03:48.593810 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:48.593777    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-qjskm_5ff415ce-f7eb-4751-997a-b39a7ac75fad/kuadrant-console-plugin/0.log"
Jul 29 22:03:48.744671 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:48.744645    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-7rcwp_8013d924-e01f-4ca5-9bde-2e5ed1fdc288/limitador/0.log"
Jul 29 22:03:50.903990 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:50.903907    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-kfcjl_251fbdae-e030-4ded-bda7-bb7e2dd09689/monitoring-plugin/0.log"
Jul 29 22:03:51.103155 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.103128    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxs7w_bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf/node-exporter/0.log"
Jul 29 22:03:51.132518 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.132495    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxs7w_bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf/kube-rbac-proxy/0.log"
Jul 29 22:03:51.157611 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.157559    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nxs7w_bb9c0cfc-21b1-4656-9b95-f6011c5e4dbf/init-textfile/0.log"
Jul 29 22:03:51.279669 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.279646    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/prometheus/0.log"
Jul 29 22:03:51.308918 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.308897    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/config-reloader/0.log"
Jul 29 22:03:51.336245 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.336225    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/thanos-sidecar/0.log"
Jul 29 22:03:51.368630 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.368607    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/kube-rbac-proxy-web/0.log"
Jul 29 22:03:51.398259 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.398237    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/kube-rbac-proxy/0.log"
Jul 29 22:03:51.425106 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.425049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/kube-rbac-proxy-thanos/0.log"
Jul 29 22:03:51.448379 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.448355    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_503bd6c7-1043-4c90-a258-6d131606cdfa/init-config-reloader/0.log"
Jul 29 22:03:51.479450 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.479425    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-4v7vr_d7714ff7-df3e-4ff7-b718-13f65bb54017/prometheus-operator/0.log"
Jul 29 22:03:51.501536 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.501515    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-4v7vr_d7714ff7-df3e-4ff7-b718-13f65bb54017/kube-rbac-proxy/0.log"
Jul 29 22:03:51.530578 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.530555    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-6wmg8_e18423f2-a537-409e-9137-983d1c2e237f/prometheus-operator-admission-webhook/0.log"
Jul 29 22:03:51.649115 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.649086    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/thanos-query/0.log"
Jul 29 22:03:51.671596 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.671572    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/kube-rbac-proxy-web/0.log"
Jul 29 22:03:51.699836 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.699782    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/kube-rbac-proxy/0.log"
Jul 29 22:03:51.722414 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.722394    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/prom-label-proxy/0.log"
Jul 29 22:03:51.744890 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.744871    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/kube-rbac-proxy-rules/0.log"
Jul 29 22:03:51.768477 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:51.768456    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97c7954bb-vkjvs_84266f12-a4b3-4d83-8528-4f101873b7ec/kube-rbac-proxy-metrics/0.log"
Jul 29 22:03:53.219679 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.219643    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"]
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.219965    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36433b9c-9ca9-41c6-b5ae-481d432022d4" containerName="payload-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.219977    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="36433b9c-9ca9-41c6-b5ae-481d432022d4" containerName="payload-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.219990    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67f73e49-73a0-4b39-89d7-333193dbd812" containerName="maas-api"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.219996    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="67f73e49-73a0-4b39-89d7-333193dbd812" containerName="maas-api"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220006    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e96de5f3-6329-4c23-a6fa-56e046ad7112" containerName="payload-pre-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220012    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e96de5f3-6329-4c23-a6fa-56e046ad7112" containerName="payload-pre-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220022    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80daedf-c89c-4db7-a6c6-fefeda62f594" containerName="payload-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220029    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80daedf-c89c-4db7-a6c6-fefeda62f594" containerName="payload-processing"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220042    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93920741-c808-44ba-abdb-e90990a66097" containerName="maas-api"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220049    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="93920741-c808-44ba-abdb-e90990a66097" containerName="maas-api"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220055    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15104415-6c19-4f89-8b3c-2bb09c25af23" containerName="main"
Jul 29 22:03:53.220059 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220060    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="15104415-6c19-4f89-8b3c-2bb09c25af23" containerName="main"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220108    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="36433b9c-9ca9-41c6-b5ae-481d432022d4" containerName="payload-processing"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220115    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="67f73e49-73a0-4b39-89d7-333193dbd812" containerName="maas-api"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220123    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e96de5f3-6329-4c23-a6fa-56e046ad7112" containerName="payload-pre-processing"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220130    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="15104415-6c19-4f89-8b3c-2bb09c25af23" containerName="main"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220137    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80daedf-c89c-4db7-a6c6-fefeda62f594" containerName="payload-processing"
Jul 29 22:03:53.220427 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.220142    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="93920741-c808-44ba-abdb-e90990a66097" containerName="maas-api"
Jul 29 22:03:53.223061 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.223042    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.225720 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.225698    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crrnx\"/\"kube-root-ca.crt\""
Jul 29 22:03:53.226585 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.226566    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-crrnx\"/\"default-dockercfg-rf2bz\""
Jul 29 22:03:53.226585 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.226581    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crrnx\"/\"openshift-service-ca.crt\""
Jul 29 22:03:53.242655 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.242632    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"]
Jul 29 22:03:53.347776 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.347753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-sys\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.347898 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.347800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-proc\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.347898 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.347821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-podres\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.347898 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.347874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-lib-modules\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.348004 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.347903    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlm4\" (UniqueName: \"kubernetes.io/projected/1cda3e60-a258-49e8-9944-a4a4ad298569-kube-api-access-jxlm4\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.448844 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-proc\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.448844 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-podres\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448870    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-lib-modules\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlm4\" (UniqueName: \"kubernetes.io/projected/1cda3e60-a258-49e8-9944-a4a4ad298569-kube-api-access-jxlm4\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-sys\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-proc\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-lib-modules\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.448995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-podres\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.449046 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.449044    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1cda3e60-a258-49e8-9944-a4a4ad298569-sys\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.456945 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.456917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlm4\" (UniqueName: \"kubernetes.io/projected/1cda3e60-a258-49e8-9944-a4a4ad298569-kube-api-access-jxlm4\") pod \"perf-node-gather-daemonset-5kjqg\" (UID: \"1cda3e60-a258-49e8-9944-a4a4ad298569\") " pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.532438 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.532412    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:53.663372 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:53.663345    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"]
Jul 29 22:03:53.665885 ip-10-0-139-241 kubenswrapper[2569]: W0729 22:03:53.665847    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1cda3e60_a258_49e8_9944_a4a4ad298569.slice/crio-15bb93274535d8e0b01c9f23c021147b5451592b8edd0fca32a177de6a7303fa WatchSource:0}: Error finding container 15bb93274535d8e0b01c9f23c021147b5451592b8edd0fca32a177de6a7303fa: Status 404 returned error can't find the container with id 15bb93274535d8e0b01c9f23c021147b5451592b8edd0fca32a177de6a7303fa
Jul 29 22:03:54.077899 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:54.077867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg" event={"ID":"1cda3e60-a258-49e8-9944-a4a4ad298569","Type":"ContainerStarted","Data":"8ec94679189e34b322f9585db9a15bc12ac70d5c50b0a493b084fa71a5ca9059"}
Jul 29 22:03:54.078080 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:54.077917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg" event={"ID":"1cda3e60-a258-49e8-9944-a4a4ad298569","Type":"ContainerStarted","Data":"15bb93274535d8e0b01c9f23c021147b5451592b8edd0fca32a177de6a7303fa"}
Jul 29 22:03:54.078080 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:54.077958    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:03:54.093591 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:54.093548    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg" podStartSLOduration=1.093537288 podStartE2EDuration="1.093537288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:03:54.092921408 +0000 UTC m=+4646.690897775" watchObservedRunningTime="2026-07-29 22:03:54.093537288 +0000 UTC m=+4646.691513645"
Jul 29 22:03:54.188377 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:54.188336    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-5j86x_cf091058-cc54-4fdf-92fc-601a36e78ea9/download-server/0.log"
Jul 29 22:03:55.820024 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:55.819993    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8v9m_9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7/dns/0.log"
Jul 29 22:03:55.842199 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:55.842169    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r8v9m_9d34bdd2-54b1-4b36-a707-a0b72a9e3ae7/kube-rbac-proxy/0.log"
Jul 29 22:03:55.957598 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:55.957574    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wx7q5_d35b2869-d157-4f59-a05e-3dd14b1ff33e/dns-node-resolver/0.log"
Jul 29 22:03:56.491376 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:56.491345    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-789894bd89-pzz4w_bee974dc-0569-424d-bfda-cfa61af808d0/registry/0.log"
Jul 29 22:03:56.542636 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:56.542592    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j96sc_fd012239-a602-4f38-a26e-ecdaad610b62/node-ca/0.log"
Jul 29 22:03:57.685691 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:57.685657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b585ddb49-whcx8_ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08/kube-auth-proxy/0.log"
Jul 29 22:03:58.684664 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:58.684636    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sfgfh_03c2ee14-ab98-46df-96f7-312d4fd0b652/serve-healthcheck-canary/0.log"
Jul 29 22:03:59.289995 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:59.289962    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75r9g_e0b4ac78-fa1b-4793-8c29-19b513f368e0/kube-rbac-proxy/0.log"
Jul 29 22:03:59.311876 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:59.311852    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75r9g_e0b4ac78-fa1b-4793-8c29-19b513f368e0/exporter/0.log"
Jul 29 22:03:59.334123 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:03:59.334101    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-75r9g_e0b4ac78-fa1b-4793-8c29-19b513f368e0/extractor/0.log"
Jul 29 22:04:00.092567 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:00.092538    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-crrnx/perf-node-gather-daemonset-5kjqg"
Jul 29 22:04:02.440088 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:02.440060    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dbc94cc7-h954v_096a549e-855d-439d-a839-755ad9b7387d/manager/0.log"
Jul 29 22:04:02.461502 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:02.461480    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7dbc94cc7-h954v_096a549e-855d-439d-a839-755ad9b7387d/copy-manifests/0.log"
Jul 29 22:04:03.896944 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:03.896912    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6896cc89b8-rbvx9_6575cb1c-f4b1-46a1-a5c6-d9f1abd8b6eb/manager/0.log"
Jul 29 22:04:04.107399 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:04.107333    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b585ddb49-whcx8_ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08/kube-auth-proxy/0.log"
Jul 29 22:04:05.754087 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:05.754059    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b585ddb49-whcx8_ffe03c35-f80e-4e05-b1cb-a3f0b1bafd08/kube-auth-proxy/0.log"
Jul 29 22:04:08.845898 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:08.845866    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-754pb_9655674a-1fb8-427a-b965-fc4c6e7c17eb/migrator/0.log"
Jul 29 22:04:08.868442 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:08.868417    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-754pb_9655674a-1fb8-427a-b965-fc4c6e7c17eb/graceful-termination/0.log"
Jul 29 22:04:10.478054 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.478032    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/kube-multus-additional-cni-plugins/0.log"
Jul 29 22:04:10.503637 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.503613    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/egress-router-binary-copy/0.log"
Jul 29 22:04:10.525826 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.525808    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/cni-plugins/0.log"
Jul 29 22:04:10.549325 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.549305    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/bond-cni-plugin/0.log"
Jul 29 22:04:10.572995 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.572975    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/routeoverride-cni/0.log"
Jul 29 22:04:10.597027 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.597009    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/whereabouts-cni-bincopy/0.log"
Jul 29 22:04:10.618599 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.618583    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-fkswq_48e16bae-b350-46ad-bd4c-c5c7e288ebf1/whereabouts-cni/0.log"
Jul 29 22:04:10.814673 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.814583    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-grzwl_17ee39ab-9224-4efb-9bc6-83ce76a3f90e/kube-multus/0.log"
Jul 29 22:04:10.968845 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.968820    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jj8sp_ad1052f7-0f92-4807-a179-014bf646b54f/network-metrics-daemon/0.log"
Jul 29 22:04:10.988700 ip-10-0-139-241 kubenswrapper[2569]: I0729 22:04:10.988674    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jj8sp_ad1052f7-0f92-4807-a179-014bf646b54f/kube-rbac-proxy/0.log"