Sep 06 04:53:41.146098 ip-10-0-130-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 06 04:53:41.146112 ip-10-0-130-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 06 04:53:41.146121 ip-10-0-130-133 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 06 04:53:41.146503 ip-10-0-130-133 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 06 04:53:51.392178 ip-10-0-130-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 06 04:53:51.392193 ip-10-0-130-133 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f3a8ad4a91f743a28f28d27347e5ef7b --
Sep 06 04:55:37.781212 ip-10-0-130-133 systemd[1]: Starting Kubernetes Kubelet...
Sep 06 04:55:38.228085 ip-10-0-130-133 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:38.228085 ip-10-0-130-133 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 06 04:55:38.228085 ip-10-0-130-133 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:38.228085 ip-10-0-130-133 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 06 04:55:38.228085 ip-10-0-130-133 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 06 04:55:38.230527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.230435    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 06 04:55:38.235237 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235215    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.235237 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235233    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.235237 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235237    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.235237 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235241    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.235237 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235244    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235247    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235250    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235253    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235255    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235258    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235261    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235263    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235266    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235268    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235271    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235273    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235276    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235279    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235281    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235284    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235286    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235289    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235291    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235293    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.235423 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235296    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235299    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235302    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235304    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235306    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235309    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235311    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235315    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235317    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235320    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235322    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235325    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235327    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235330    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235334    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235338    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235340    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235343    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235346    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235348    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.235969 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235351    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235353    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235355    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235358    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235360    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235363    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235366    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235369    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235371    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235374    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235376    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235379    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235381    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235384    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235386    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235389    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235391    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235394    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235396    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235399    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.236465 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235401    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235404    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235406    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235409    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235411    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235415    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235418    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235421    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235424    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235427    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235430    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235434    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235438    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235442    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235446    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235448    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235452    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235454    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235457    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.236995 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235461    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235463    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235466    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235468    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235471    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235473    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235476    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235478    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235481    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235484    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235486    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235489    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235492    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235496    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235499    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.235503    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236242    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236249    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236253    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.237454 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236257    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236260    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236263    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236266    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236269    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236272    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236275    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236277    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236280    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236282    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236285    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236288    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236290    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236292    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236297    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236300    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236303    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236305    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236308    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.238013 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236311    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236313    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236316    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236318    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236321    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236323    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236326    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236328    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236331    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236333    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236336    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236338    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236341    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236345    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236348    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236350    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236353    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236356    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236358    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236361    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.238528 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236363    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236366    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236369    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236371    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236374    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236376    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236379    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236381    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236385    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236387    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236390    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236392    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236395    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236398    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236400    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236402    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236405    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236407    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236410    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236412    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.239034 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236414    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236418    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236422    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236425    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236429    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236431    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236434    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236437    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236439    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236442    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236444    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236447    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236449    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236452    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236454    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236457    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236459    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236462    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236464    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236466    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.239535 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236469    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236472    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236475    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236477    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236488    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236491    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236494    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236497    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236500    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236502    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236506    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236509    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236511    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236514    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236517    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236520    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236522    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236958    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236964    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236967    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.240038 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236970    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236973    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236976    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236979    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236982    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236984    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236987    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236990    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236992    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236994    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.236998    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237001    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237004    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237008    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237010    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237013    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237016    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237018    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237021    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237023    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.240541 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237025    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237029    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237031    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237034    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237036    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237038    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237041    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237043    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237047    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237049    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237052    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237055    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237057    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237061    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237065    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237067    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237070    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237074    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237079    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.241054 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237083    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237086    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237088    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237091    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237094    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237097    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237100    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237103    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237106    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237108    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237111    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237113    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237116    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237118    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237121    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237124    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237126    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237129    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237131    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237134    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.241510 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237136    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237139    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237141    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237144    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237146    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237149    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237151    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237156    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237158    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237161    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237163    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237166    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237170    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237173    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237176    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237179    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237182    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237184    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237187    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.242018 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237190    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237194    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237198    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237200    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237203    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237206    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237208    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237213    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237216    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237218    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237221    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237223    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237226    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237229    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237231    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237234    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237236    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237239    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237306    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237339    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 06 04:55:38.242476 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237345    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237350    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237354    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237357    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237361    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237365    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237368    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237371    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237375    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237378    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237381    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237384    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237387    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237390    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237400    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237403    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237408    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237411    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237414    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237418    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237421    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237424    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237427    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 06 04:55:38.242990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237431    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237434    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237439    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237442    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237445    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237448    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237451    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237455    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237458    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237461    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237464    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237467    2576 flags.go:64] FLAG: --enable-server="true"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237469    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237473    2576 flags.go:64] FLAG: --event-burst="100"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237476    2576 flags.go:64] FLAG: --event-qps="50"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237479    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237482    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237485    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237488    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237491    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237494    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237497    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237499    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237502    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237506    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 06 04:55:38.243533 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237509    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237512    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237514    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237518    2576 flags.go:64] FLAG: --feature-gates=""
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237522    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237524    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237527    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237530    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237533    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237537    2576 flags.go:64] FLAG: --help="false"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237540    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237543    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237546    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237549    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237552    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237555    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237558    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237561    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237564    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237566    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237569    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237572    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237575    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237577    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 06 04:55:38.244226 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237580    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237583    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237586    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237588    2576 flags.go:64] FLAG: --lock-file=""
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237591    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237594    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237597    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237602    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237606    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237609    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237611    2576 flags.go:64] FLAG: --logging-format="text"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237615    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237618    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237621    2576 flags.go:64] FLAG: --manifest-url=""
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237623    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237627    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237630    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237636    2576 flags.go:64] FLAG: --max-pods="110"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237638    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237641    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237644    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237647    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237650    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237653    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237656    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 06 04:55:38.244836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237663    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237666    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237669    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237672    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237675    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237680    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237683    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237686    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237689    2576 flags.go:64] FLAG: --port="10250"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237692    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237695    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-073413794a60fd778"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237698    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237701    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237703    2576 flags.go:64] FLAG: --register-node="true"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237706    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237710    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237713    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237716    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237719    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237724    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237727    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237730    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237732    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237735    2576 flags.go:64] FLAG: --runonce="false"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237739    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 06 04:55:38.245456 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237742    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237745    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237748    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237750    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237753    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237756    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237759    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237762    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237764    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237768    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237770    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237773    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237776    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237778    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237783    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237786    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237789    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237796    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237798    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237801    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237804    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237806    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237809    2576 flags.go:64] FLAG: --v="2"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237813    2576 flags.go:64] FLAG: --version="false"
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237817    2576 flags.go:64] FLAG: --vmodule=""
Sep 06 04:55:38.246081 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237821    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.237826    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237953    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237958    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237960    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237964    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237968    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237970    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237973    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237976    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237979    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237982    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237984    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237988    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237992    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237995    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.237999    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238002    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238004    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238007    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.246683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238009    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238013    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238015    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238018    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238021    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238023    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238026    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238028    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238031    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238034    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238036    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238038    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238041    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238045    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238048    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238050    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238053    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238056    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238060    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.247198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238062    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238065    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238067    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238070    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238072    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238075    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238077    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238080    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238082    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238085    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238087    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238090    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238092    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238095    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238097    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238100    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238102    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238105    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238107    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238109    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.247698 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238112    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238114    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238117    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238119    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238122    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238124    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238128    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238130    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238133    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238137    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238141    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238146    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238150    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238153    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238156    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238160    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238163    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238166    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238169    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238172    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.248204 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238175    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238178    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238180    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238183    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238186    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238188    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238192    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238196    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238198    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238201    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238204    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238206    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238209    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238211    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238214    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238217    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238219    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238222    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238225    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.248696 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238228    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238231    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.238234    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.238242    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.245590    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.245608    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.245630    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.245636    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245692    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245697    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245702    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245706    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245709    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245712    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.249236 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245715    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245717    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245720    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245723    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245726    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245728    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245731    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245734    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245736    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245739    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245741    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245744    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245746    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245749    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245751    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245754    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245756    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245759    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245761    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245764    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.249612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245766    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245769    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245771    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245775    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245779    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245783    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245786    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245789    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245791    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245794    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245797    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245799    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245803    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245805    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245808    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245811    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245813    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245816    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245818    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245820    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.250283 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245823    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245826    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245828    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245831    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245833    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245836    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245838    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245842    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245845    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245848    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245850    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245853    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245855    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245858    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245860    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245863    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245866    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245868    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245871    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.250779 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245873    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245876    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245878    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245880    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245883    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245885    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245887    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245890    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245892    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245895    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245917    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245921    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245924    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245926    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245928    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245931    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245934    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245936    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245939    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245941    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.251299 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245944    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245946    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245949    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245951    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245953    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245956    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245958    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245962    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245964    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245966    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245970    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245973    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245975    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.245978    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.245983    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:38.251780 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246112    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246118    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246121    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246123    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246126    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246129    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246132    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246134    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246137    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246139    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246141    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246144    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246146    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246149    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246151    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246153    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246156    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246158    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246161    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246163    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 06 04:55:38.252179 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246166    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246168    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246170    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246173    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246176    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246178    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246181    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246183    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246187    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246190    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246192    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246195    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246197    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246199    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246202    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246204    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246206    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246209    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246211    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246214    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 06 04:55:38.252683 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246216    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246218    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246221    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246223    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246226    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246228    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246231    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246235    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246239    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246242    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246246    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246249    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246251    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246255    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246259    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246262    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246265    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246268    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246270    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 06 04:55:38.253198 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246285    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246289    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246293    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246296    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246299    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246302    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246305    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246307    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246310    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246313    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246315    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246318    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246320    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246323    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246325    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246328    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246330    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246332    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246335    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246337    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 06 04:55:38.253655 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246340    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246342    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246345    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246347    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246350    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246352    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246354    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246357    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246359    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246362    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246365    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246367    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246369    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246372    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246374    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246377    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246380    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246382    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246385    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 06 04:55:38.254241 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:38.246387    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.246392    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.246500    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.248690    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.249689    2576 server.go:1019] "Starting client certificate rotation"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.249801    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.249843    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.252547    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 04:55:38.254915 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.254891    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 06 04:55:38.264937 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.264912    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 06 04:55:38.267734 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.267717    2576 log.go:25] "Validated CRI v1 image API"
Sep 06 04:55:38.268657 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.268639    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 06 04:55:38.271457 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.271433    2576 fs.go:135] Filesystem UUIDs: map[5b26dac3-379c-4d7c-90fa-c99272c6976a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d6536a0b-e3bf-4fcf-aaee-0d213e288310:/dev/nvme0n1p3]
Sep 06 04:55:38.271527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.271457    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 06 04:55:38.276511 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.276392    2576 manager.go:216] Machine: {Timestamp:2026-09-06 04:55:38.275033891 +0000 UTC m=+0.364111574 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099687 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2da3d645a3c7dd84778d4c806f667c SystemUUID:ec2da3d6-45a3-c7dd-8477-8d4c806f667c BootID:f3a8ad4a-91f7-43a2-8f28-d27347e5ef7b Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:25:d3:5a:69 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:25:d3:5a:69 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:53:a3:79:1a:e2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 06 04:55:38.276511 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.276509    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 06 04:55:38.276614 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.276591    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 06 04:55:38.279374 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.279343    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 06 04:55:38.279526 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.279381    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 06 04:55:38.280073 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.280062    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 06 04:55:38.280110 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.280076    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 06 04:55:38.280110 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.280089    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 04:55:38.280776 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.280750    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 06 04:55:38.280970 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.280956    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 04:55:38.282266 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.282254    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 06 04:55:38.285891 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.285879    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 04:55:38.286059 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286048    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 06 04:55:38.286093 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286081    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 06 04:55:38.286129 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286095    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 06 04:55:38.286129 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286105    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 06 04:55:38.286129 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286114    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 06 04:55:38.286129 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286123    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 06 04:55:38.286496 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286485    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 04:55:38.286542 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.286503    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 06 04:55:38.288934 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.288915    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 06 04:55:38.291665 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291648    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 06 04:55:38.291733 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291682    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 06 04:55:38.291914 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291882    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 06 04:55:38.291952 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291922    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 06 04:55:38.291952 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291930    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 06 04:55:38.291952 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291935    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 06 04:55:38.291952 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291944    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 06 04:55:38.291952 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.291950    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 06 04:55:38.292499 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292490    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 06 04:55:38.292529 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292500    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 06 04:55:38.292529 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292507    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 06 04:55:38.292529 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292512    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 06 04:55:38.292608 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292532    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 06 04:55:38.292608 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292545    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 06 04:55:38.292608 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292573    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 06 04:55:38.292608 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.292578    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 06 04:55:38.297548 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.297513    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 06 04:55:38.297642 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.297545    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 06 04:55:38.297642 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.297599    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 06 04:55:38.298235 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.298219    2576 server.go:1268] "Started kubelet"
Sep 06 04:55:38.298336 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.298295    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 06 04:55:38.298397 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.298344    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 06 04:55:38.298432 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.298414    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 06 04:55:38.299816 ip-10-0-130-133 systemd[1]: Started Kubernetes Kubelet.
Sep 06 04:55:38.299957 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.299942    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 06 04:55:38.300466 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.300451    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 06 04:55:38.302564 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.302400    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q7x86"
Sep 06 04:55:38.302687 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.302645    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 06 04:55:38.305369 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.305356    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 06 04:55:38.305416 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.305367    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 06 04:55:38.305470 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.305442    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.306105 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306089    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 06 04:55:38.306223 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306106    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 06 04:55:38.306223 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306226    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 06 04:55:38.306374 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306285    2576 factory.go:55] Registering systemd factory
Sep 06 04:55:38.306374 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306302    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 06 04:55:38.306374 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306330    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 06 04:55:38.306374 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306355    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 06 04:55:38.306578 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306503    2576 factory.go:153] Registering CRI-O factory
Sep 06 04:55:38.306578 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306514    2576 factory.go:223] Registration of the crio container factory successfully
Sep 06 04:55:38.306673 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306584    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 06 04:55:38.306673 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306613    2576 factory.go:103] Registering Raw factory
Sep 06 04:55:38.306673 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.306627    2576 manager.go:1196] Started watching for new ooms in manager
Sep 06 04:55:38.307285 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.306075    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-133.ec2.internal.18d2a260b777c5ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-133.ec2.internal,UID:ip-10-0-130-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-133.ec2.internal,},FirstTimestamp:2026-09-06 04:55:38.29814827 +0000 UTC m=+0.387225952,LastTimestamp:2026-09-06 04:55:38.29814827 +0000 UTC m=+0.387225952,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-133.ec2.internal,}"
Sep 06 04:55:38.307397 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.307309    2576 manager.go:349] Starting recovery of all containers
Sep 06 04:55:38.309608 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.309588    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q7x86"
Sep 06 04:55:38.313325 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.313299    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 06 04:55:38.313419 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.313395    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 06 04:55:38.317828 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.317812    2576 manager.go:354] Recovery completed
Sep 06 04:55:38.321227 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.321183    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 06 04:55:38.324555 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.324540    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.326229 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326213    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.326315 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326247    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.326315 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326261    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.326857 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326842    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 06 04:55:38.326857 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326857    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 06 04:55:38.326993 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326858    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.326993 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.326873    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 06 04:55:38.327537 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.327522    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.327580 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.327549    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.327580 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.327563    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.328589 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.328578    2576 policy_none.go:49] "None policy: Start"
Sep 06 04:55:38.328633 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.328593    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 06 04:55:38.328633 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.328603    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 06 04:55:38.330272 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.330258    2576 policy_none.go:47] "Start"
Sep 06 04:55:38.368541 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.368524    2576 manager.go:337] "Starting Device Plugin manager"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.368575    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.368588    2576 server.go:87] "Starting device plugin registration server"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.369025    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.369036    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.369216    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.369328    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.369339    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.369728    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.369770    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.377654    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.379014    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.379055    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.379083    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.379128    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 06 04:55:38.388325 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.381117    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:38.469499 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.469467    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.470510 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.470494    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.470582 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.470530    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.470582 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.470542    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.470647 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.470583    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.479122 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.479044    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.479122 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.479076    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-133.ec2.internal\": node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.479229 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.479216    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"]
Sep 06 04:55:38.479290 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.479280    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.480494 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.480472    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.480567 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.480510    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.480567 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.480524    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.482457 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.482445    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.482545 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.482531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.482580 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.482570    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.483327 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483306    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.483422 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483335    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.483422 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483351    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.483422 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483360    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.483422 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483384    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.483422 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.483396    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.485319 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.485303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.485361 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.485339    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 06 04:55:38.486260 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.486241    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientMemory"
Sep 06 04:55:38.486338 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.486275    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasNoDiskPressure"
Sep 06 04:55:38.486338 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.486289    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeHasSufficientPID"
Sep 06 04:55:38.496004 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.495983    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.508065 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.508042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.508160 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.508066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f9cfc38a9d54d6881403da441b0a834-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"6f9cfc38a9d54d6881403da441b0a834\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.508160 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.508086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.511714 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.511696    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-133.ec2.internal\" not found" node="ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.516138 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.516122    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-133.ec2.internal\" not found" node="ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.597051 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.597016    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.608292 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.608292 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.608493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f9cfc38a9d54d6881403da441b0a834-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"6f9cfc38a9d54d6881403da441b0a834\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.608493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.608493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f9cfc38a9d54d6881403da441b0a834-config\") pod \"kube-apiserver-proxy-ip-10-0-130-133.ec2.internal\" (UID: \"6f9cfc38a9d54d6881403da441b0a834\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.608493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.608374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dff361b9784023f163739c4d30ad35d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal\" (UID: \"1dff361b9784023f163739c4d30ad35d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.697410 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.697377    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.798212 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.798133    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.815541 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.815512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.819420 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:38.819398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:38.898961 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.898922    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:38.999540 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:38.999507    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.100160 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.100090    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.200615 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.200577    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.241620 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.241585    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:39.250244 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.250221    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 06 04:55:39.250381 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.250363    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 04:55:39.250430 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.250363    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 06 04:55:39.301175 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.301148    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.312200 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.312153    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-05 04:50:38 +0000 UTC" deadline="2028-05-05 15:26:08.957892697 +0000 UTC"
Sep 06 04:55:39.312200 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.312195    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14578h30m29.645701446s"
Sep 06 04:55:39.401248 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.401213    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.401951 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:39.401921    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9cfc38a9d54d6881403da441b0a834.slice/crio-a1d0a2a26acbeb0882b5f73f54e2349dbf5e0e681e76a53e538822f542e50e43 WatchSource:0}: Error finding container a1d0a2a26acbeb0882b5f73f54e2349dbf5e0e681e76a53e538822f542e50e43: Status 404 returned error can't find the container with id a1d0a2a26acbeb0882b5f73f54e2349dbf5e0e681e76a53e538822f542e50e43
Sep 06 04:55:39.402158 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:39.402136    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dff361b9784023f163739c4d30ad35d.slice/crio-58aa2cc8af799f64303301741b1563b7fd507567253f400ae9282989ef3ab274 WatchSource:0}: Error finding container 58aa2cc8af799f64303301741b1563b7fd507567253f400ae9282989ef3ab274: Status 404 returned error can't find the container with id 58aa2cc8af799f64303301741b1563b7fd507567253f400ae9282989ef3ab274
Sep 06 04:55:39.408103 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.408087    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 04:55:39.501530 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.501480    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.602055 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:39.602024    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-133.ec2.internal\" not found"
Sep 06 04:55:39.645832 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.645777    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:39.706210 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.706182    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:39.716529 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.716505    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 04:55:39.717554 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.717538    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal"
Sep 06 04:55:39.725754 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.725739    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 06 04:55:39.857536 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:39.857504    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:40.287191 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.287152    2576 apiserver.go:52] "Watching apiserver"
Sep 06 04:55:40.295232 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.295204    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 06 04:55:40.295614 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.295580    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-pjrd9","openshift-multus/network-metrics-daemon-r7t92","openshift-network-operator/iptables-alerter-9cb22","kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal","openshift-cluster-node-tuning-operator/tuned-plp8r","openshift-dns/node-resolver-jwsh9","openshift-image-registry/node-ca-rgc5h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal","openshift-multus/multus-7k4dl","openshift-network-diagnostics/network-check-target-nfv2z","openshift-ovn-kubernetes/ovnkube-node-pmmpb","kube-system/global-pull-secret-syncer-8vxsx","kube-system/konnectivity-agent-rkzgd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"]
Sep 06 04:55:40.298787 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.298456    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:40.298787 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.298542    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:40.300742 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.300718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.303200 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.303181    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 06 04:55:40.303200 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.303189    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5xbtf\""
Sep 06 04:55:40.303433 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.303417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 06 04:55:40.303502 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.303492    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.305585 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.305566    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 06 04:55:40.305918 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.305889    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.306022 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.305987    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jmthr\""
Sep 06 04:55:40.306401 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.306217    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.306401 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.306229    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 06 04:55:40.306558 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.306420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.309011 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.308633    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.309093 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.309078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 06 04:55:40.309883 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.309361    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7b59q\""
Sep 06 04:55:40.309883 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.309549    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.309883 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.309624    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.312017 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.311993    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.312438 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.312408    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.312656 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.312637    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4tnz5\""
Sep 06 04:55:40.313508 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.313488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.315891 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.315688    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.315994 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.315954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.316063 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.316007    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xtxlb\""
Sep 06 04:55:40.316292 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.316272    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.316767 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.316748    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318306    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318403    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318727    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9b6h\" (UniqueName: \"kubernetes.io/projected/9122d1d8-9c3f-412c-9350-f380c015475b-kube-api-access-n9b6h\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318751    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-registration-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91727639-16b7-49f3-9277-5a6806eb3237-agent-certs\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9122d1d8-9c3f-412c-9350-f380c015475b-iptables-alerter-script\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-device-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-etc-selinux\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.318934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-sys-fs\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319053    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-chmnx\""
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kube-api-access-9z2rj\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-socket-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9122d1d8-9c3f-412c-9350-f380c015475b-host-slash\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:40.319400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.319199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91727639-16b7-49f3-9277-5a6806eb3237-konnectivity-ca\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.320807 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.320655    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.321493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.321061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.321493 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.321129    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:40.321493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.321253    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.321493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.321355    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lxvk4\""
Sep 06 04:55:40.321493 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.321415    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 06 04:55:40.325619 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.325594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.328112 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.328086    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-425w4\""
Sep 06 04:55:40.328213 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.328154    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 06 04:55:40.329779 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.328802    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 06 04:55:40.329779 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.329001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.331450 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.331426    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 06 04:55:40.331542 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.331470    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 06 04:55:40.331708 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.331685    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7sjxc\""
Sep 06 04:55:40.331765 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.331734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.331840 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.331816    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:40.331967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.331942    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 06 04:55:40.332029 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.332011    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 06 04:55:40.333028 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.333000    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 06 04:55:40.333233 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.333218    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 06 04:55:40.336213 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.336183    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6b64x"
Sep 06 04:55:40.344099 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.344078    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6b64x"
Sep 06 04:55:40.384693 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.384647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" event={"ID":"6f9cfc38a9d54d6881403da441b0a834","Type":"ContainerStarted","Data":"a1d0a2a26acbeb0882b5f73f54e2349dbf5e0e681e76a53e538822f542e50e43"}
Sep 06 04:55:40.386336 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.386284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"1dff361b9784023f163739c4d30ad35d","Type":"ContainerStarted","Data":"58aa2cc8af799f64303301741b1563b7fd507567253f400ae9282989ef3ab274"}
Sep 06 04:55:40.407917 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.407878    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 06 04:55:40.419442 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-multus-certs\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.419586 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.419586 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-systemd\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.419586 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-log-socket\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.419586 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-kubelet\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlnw\" (UniqueName: \"kubernetes.io/projected/544f66cd-a01a-480b-aea0-23c82fc314c0-kube-api-access-drlnw\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-systemd-units\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-system-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419743    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-run\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-host\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.419794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a452e6b1-85c2-485d-8d50-3434e01caae4-host\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-slash\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-node-log\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-tmp\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544f66cd-a01a-480b-aea0-23c82fc314c0-tmp-dir\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-ovn\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.419991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovn-node-metrics-cert\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-multus-daemon-config\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9122d1d8-9c3f-412c-9350-f380c015475b-iptables-alerter-script\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kube-api-access-9z2rj\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.420194 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-var-lib-kubelet\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-kubelet-config\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-conf-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-etc-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420304    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-cni-binary-copy\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfwrw\" (UniqueName: \"kubernetes.io/projected/4d7bc33c-3588-4f47-8356-5be260b42ec6-kube-api-access-bfwrw\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-netd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-socket-dir-parent\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sft9m\" (UniqueName: \"kubernetes.io/projected/7909525c-4e27-41e3-881f-8f7127d0b124-kube-api-access-sft9m\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-registration-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-netns\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420480    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-multus\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-device-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-etc-selinux\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-sys-fs\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.420766 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420602    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-system-cni-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-tuned\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420668    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-script-lib\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2pkz\" (UniqueName: \"kubernetes.io/projected/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-kube-api-access-g2pkz\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-bin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-sys\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-config\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9122d1d8-9c3f-412c-9350-f380c015475b-iptables-alerter-script\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.420892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-registration-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-sys-fs\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-device-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-etc-selinux\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-env-overrides\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-netns\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9122d1d8-9c3f-412c-9350-f380c015475b-host-slash\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.421473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91727639-16b7-49f3-9277-5a6806eb3237-konnectivity-ca\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9122d1d8-9c3f-412c-9350-f380c015475b-host-slash\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysconfig\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-966pt\" (UniqueName: \"kubernetes.io/projected/4f3f6321-5aaf-4656-ad4c-04109aeba114-kube-api-access-966pt\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91727639-16b7-49f3-9277-5a6806eb3237-agent-certs\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-modprobe-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/544f66cd-a01a-480b-aea0-23c82fc314c0-hosts-file\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfrw2\" (UniqueName: \"kubernetes.io/projected/a452e6b1-85c2-485d-8d50-3434e01caae4-kube-api-access-qfrw2\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-k8s-cni-cncf-io\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421727    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-etc-kubernetes\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-conf\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-lib-modules\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-kubelet\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-cnibin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.422119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-socket-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421888    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-var-lib-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-cnibin\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.421977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwsr\" (UniqueName: \"kubernetes.io/projected/2757f210-9c9b-4762-8631-9348f13dc03e-kube-api-access-nvwsr\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-kubernetes\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91727639-16b7-49f3-9277-5a6806eb3237-konnectivity-ca\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a452e6b1-85c2-485d-8d50-3434e01caae4-serviceca\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-systemd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422119    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-hostroot\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51ee04e0-3c86-4c9a-902e-e9040d8fa344-socket-dir\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9b6h\" (UniqueName: \"kubernetes.io/projected/9122d1d8-9c3f-412c-9350-f380c015475b-kube-api-access-n9b6h\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422222    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-bin\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.422998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422270    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.423697 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422322    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 06 04:55:40.423697 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.422972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-dbus\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.425169 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.425109    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:40.425169 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.425133    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:40.425169 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.425146    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:40.425411 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.425208    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:40.925187912 +0000 UTC m=+3.014265589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:40.427768 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.427743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91727639-16b7-49f3-9277-5a6806eb3237-agent-certs\") pod \"konnectivity-agent-rkzgd\" (UID: \"91727639-16b7-49f3-9277-5a6806eb3237\") " pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.428732 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.428704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2rj\" (UniqueName: \"kubernetes.io/projected/51ee04e0-3c86-4c9a-902e-e9040d8fa344-kube-api-access-9z2rj\") pod \"aws-ebs-csi-driver-node-l28nr\" (UID: \"51ee04e0-3c86-4c9a-902e-e9040d8fa344\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.430712 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.430690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9b6h\" (UniqueName: \"kubernetes.io/projected/9122d1d8-9c3f-412c-9350-f380c015475b-kube-api-access-n9b6h\") pod \"iptables-alerter-9cb22\" (UID: \"9122d1d8-9c3f-412c-9350-f380c015475b\") " pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.448875 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.448845    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:40.523615 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfwrw\" (UniqueName: \"kubernetes.io/projected/4d7bc33c-3588-4f47-8356-5be260b42ec6-kube-api-access-bfwrw\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.523615 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-netd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-socket-dir-parent\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-netd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sft9m\" (UniqueName: \"kubernetes.io/projected/7909525c-4e27-41e3-881f-8f7127d0b124-kube-api-access-sft9m\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-netns\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-socket-dir-parent\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-multus\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.523874 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-netns\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-system-cni-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-tuned\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-multus\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-script-lib\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.523989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-system-cni-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2pkz\" (UniqueName: \"kubernetes.io/projected/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-kube-api-access-g2pkz\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.524035    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-bin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-sys\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.524106    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.024085599 +0000 UTC m=+3.113163290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-config\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-sys\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-cni-bin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-env-overrides\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-netns\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysconfig\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.524313 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-966pt\" (UniqueName: \"kubernetes.io/projected/4f3f6321-5aaf-4656-ad4c-04109aeba114-kube-api-access-966pt\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-modprobe-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-netns\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/544f66cd-a01a-480b-aea0-23c82fc314c0-hosts-file\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524324    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfrw2\" (UniqueName: \"kubernetes.io/projected/a452e6b1-85c2-485d-8d50-3434e01caae4-kube-api-access-qfrw2\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-k8s-cni-cncf-io\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysconfig\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-etc-kubernetes\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-etc-kubernetes\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-conf\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-lib-modules\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-kubelet\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524526    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-cnibin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-modprobe-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-var-lib-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-env-overrides\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-cnibin\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.525224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-conf\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwsr\" (UniqueName: \"kubernetes.io/projected/2757f210-9c9b-4762-8631-9348f13dc03e-kube-api-access-nvwsr\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-script-lib\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-cnibin\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-cnibin\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-var-lib-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-lib-modules\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/544f66cd-a01a-480b-aea0-23c82fc314c0-hosts-file\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-kubernetes\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.524744    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-kubelet\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.524789    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.024777366 +0000 UTC m=+3.113855039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524827    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a452e6b1-85c2-485d-8d50-3434e01caae4-serviceca\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-systemd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-k8s-cni-cncf-io\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.526278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-hostroot\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-kubernetes\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-systemd\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.524992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-hostroot\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-bin\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-cni-bin\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-dbus\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-multus-certs\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525149    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-systemd\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-dbus\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525214    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-log-socket\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.527347 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-log-socket\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-kubelet\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a452e6b1-85c2-485d-8d50-3434e01caae4-serviceca\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-sysctl-d\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-systemd\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-run-multus-certs\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlnw\" (UniqueName: \"kubernetes.io/projected/544f66cd-a01a-480b-aea0-23c82fc314c0-kube-api-access-drlnw\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-host-var-lib-kubelet\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-systemd-units\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovnkube-config\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-system-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-systemd-units\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-cni-binary-copy\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-system-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-run\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.528264 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-host\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a452e6b1-85c2-485d-8d50-3434e01caae4-host\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-run\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-slash\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-node-log\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2757f210-9c9b-4762-8631-9348f13dc03e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-tmp\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-host-slash\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544f66cd-a01a-480b-aea0-23c82fc314c0-tmp-dir\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-host\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a452e6b1-85c2-485d-8d50-3434e01caae4-host\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-ovn\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovn-node-metrics-cert\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-multus-daemon-config\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-var-lib-kubelet\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.525975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-kubelet-config\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.529193 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-conf-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/544f66cd-a01a-480b-aea0-23c82fc314c0-tmp-dir\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-etc-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-cni-binary-copy\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-node-log\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-cni-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d7bc33c-3588-4f47-8356-5be260b42ec6-var-lib-kubelet\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7909525c-4e27-41e3-881f-8f7127d0b124-multus-conf-dir\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526168    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-etc-openvswitch\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526202    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-kubelet-config\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526211    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-run-ovn\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-cni-binary-copy\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.526781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7909525c-4e27-41e3-881f-8f7127d0b124-multus-daemon-config\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.527006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-etc-tuned\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.527032    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2757f210-9c9b-4762-8631-9348f13dc03e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.528542    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d7bc33c-3588-4f47-8356-5be260b42ec6-tmp\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.530133 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.528970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-ovn-node-metrics-cert\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.537115 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.537083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfwrw\" (UniqueName: \"kubernetes.io/projected/4d7bc33c-3588-4f47-8356-5be260b42ec6-kube-api-access-bfwrw\") pod \"tuned-plp8r\" (UID: \"4d7bc33c-3588-4f47-8356-5be260b42ec6\") " pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.537332 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.537157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwsr\" (UniqueName: \"kubernetes.io/projected/2757f210-9c9b-4762-8631-9348f13dc03e-kube-api-access-nvwsr\") pod \"multus-additional-cni-plugins-pjrd9\" (UID: \"2757f210-9c9b-4762-8631-9348f13dc03e\") " pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.537332 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.537182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfrw2\" (UniqueName: \"kubernetes.io/projected/a452e6b1-85c2-485d-8d50-3434e01caae4-kube-api-access-qfrw2\") pod \"node-ca-rgc5h\" (UID: \"a452e6b1-85c2-485d-8d50-3434e01caae4\") " pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.537695 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.537675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2pkz\" (UniqueName: \"kubernetes.io/projected/d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c-kube-api-access-g2pkz\") pod \"ovnkube-node-pmmpb\" (UID: \"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.537787 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.537678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sft9m\" (UniqueName: \"kubernetes.io/projected/7909525c-4e27-41e3-881f-8f7127d0b124-kube-api-access-sft9m\") pod \"multus-7k4dl\" (UID: \"7909525c-4e27-41e3-881f-8f7127d0b124\") " pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.538341 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.538314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-966pt\" (UniqueName: \"kubernetes.io/projected/4f3f6321-5aaf-4656-ad4c-04109aeba114-kube-api-access-966pt\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:40.539413 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.539390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlnw\" (UniqueName: \"kubernetes.io/projected/544f66cd-a01a-480b-aea0-23c82fc314c0-kube-api-access-drlnw\") pod \"node-resolver-jwsh9\" (UID: \"544f66cd-a01a-480b-aea0-23c82fc314c0\") " pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.612636 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.612598    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:55:40.623589 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.623562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9cb22"
Sep 06 04:55:40.634783 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.634747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr"
Sep 06 04:55:40.642022 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.641994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-plp8r"
Sep 06 04:55:40.653096 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.653061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jwsh9"
Sep 06 04:55:40.659097 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.659070    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7k4dl"
Sep 06 04:55:40.669041 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.669013    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rgc5h"
Sep 06 04:55:40.675886 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.675865    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pjrd9"
Sep 06 04:55:40.681920 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.681881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:55:40.929786 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:40.929696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:40.929964 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.929897    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:40.929964 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.929941    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:40.929964 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.929954    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:40.930085 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:40.930019    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:41.92999899 +0000 UTC m=+4.019076663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:41.030887 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.030854    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:41.030887 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.030896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:41.031106 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.031005    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:41.031106 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.031005    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:41.031106 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.031061    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:42.031045056 +0000 UTC m=+4.120122743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:41.031106 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.031076    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:55:42.03106964 +0000 UTC m=+4.120147313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:41.347818 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.347714    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 04:50:40 +0000 UTC" deadline="2028-02-15 12:58:10.892867565 +0000 UTC"
Sep 06 04:55:41.347818 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.347756    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12656h2m29.545116531s"
Sep 06 04:55:41.380392 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.380359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:41.380553 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.380501    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:41.449476 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:41.449444    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda452e6b1_85c2_485d_8d50_3434e01caae4.slice/crio-a771d369500b9e14fa75b5010faf8b68ca9c5aa4e3219410c4502b06976bd392 WatchSource:0}: Error finding container a771d369500b9e14fa75b5010faf8b68ca9c5aa4e3219410c4502b06976bd392: Status 404 returned error can't find the container with id a771d369500b9e14fa75b5010faf8b68ca9c5aa4e3219410c4502b06976bd392
Sep 06 04:55:41.456693 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:41.456659    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9122d1d8_9c3f_412c_9350_f380c015475b.slice/crio-f395d0a57e8468529d04900d34ab1b717b66de66e0b05b7aaf0486953abbb729 WatchSource:0}: Error finding container f395d0a57e8468529d04900d34ab1b717b66de66e0b05b7aaf0486953abbb729: Status 404 returned error can't find the container with id f395d0a57e8468529d04900d34ab1b717b66de66e0b05b7aaf0486953abbb729
Sep 06 04:55:41.458191 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:41.458029    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544f66cd_a01a_480b_aea0_23c82fc314c0.slice/crio-c162cf66facaab0e46316667700c3b7867b3ca40dca4fb5fd62bb141896c4119 WatchSource:0}: Error finding container c162cf66facaab0e46316667700c3b7867b3ca40dca4fb5fd62bb141896c4119: Status 404 returned error can't find the container with id c162cf66facaab0e46316667700c3b7867b3ca40dca4fb5fd62bb141896c4119
Sep 06 04:55:41.461017 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:55:41.460968    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e510ba_cb0a_4e68_8ea6_39fb8ba3f87c.slice/crio-89ee4de97c9db204e1ce27b68da57bce8168ac2351c2a1e67d5c5329f7a5f805 WatchSource:0}: Error finding container 89ee4de97c9db204e1ce27b68da57bce8168ac2351c2a1e67d5c5329f7a5f805: Status 404 returned error can't find the container with id 89ee4de97c9db204e1ce27b68da57bce8168ac2351c2a1e67d5c5329f7a5f805
Sep 06 04:55:41.937391 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:41.937165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:41.937391 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.937372    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:41.937391 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.937404    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:41.937712 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.937419    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:41.937712 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:41.937486    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:43.937465626 +0000 UTC m=+6.026543303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:42.038399 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.038272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:42.038399 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.038344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:42.038647 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.038525    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:42.038647 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.038596    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:44.03857537 +0000 UTC m=+6.127653049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:42.038763 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.038687    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:42.038763 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.038727    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:55:44.038713602 +0000 UTC m=+6.127791279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:42.264774 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.264739    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 06 04:55:42.348136 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.348005    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-05 04:50:40 +0000 UTC" deadline="2028-03-31 16:06:07.407495098 +0000 UTC"
Sep 06 04:55:42.348136 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.348079    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13739h10m25.059420343s"
Sep 06 04:55:42.380599 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.380566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:42.380766 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.380711    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:42.381224 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.381203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:42.381354 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:42.381327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:42.403625 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.403585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" event={"ID":"6f9cfc38a9d54d6881403da441b0a834","Type":"ContainerStarted","Data":"048d2c2e9134ed532d1dac49ce05957d3b84ac405a9285163a50a7d298332967"}
Sep 06 04:55:42.417579 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.417433    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-133.ec2.internal" podStartSLOduration=2.417373882 podStartE2EDuration="2.417373882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:42.417108792 +0000 UTC m=+4.506186489" watchObservedRunningTime="2026-09-06 04:55:42.417373882 +0000 UTC m=+4.506451579"
Sep 06 04:55:42.425609 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.425523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-plp8r" event={"ID":"4d7bc33c-3588-4f47-8356-5be260b42ec6","Type":"ContainerStarted","Data":"5331eaea9a5e5bdc582305e6f521c6e9e10fd32ed4268c5293dff569f7794abd"}
Sep 06 04:55:42.435227 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.435184    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jwsh9" event={"ID":"544f66cd-a01a-480b-aea0-23c82fc314c0","Type":"ContainerStarted","Data":"c162cf66facaab0e46316667700c3b7867b3ca40dca4fb5fd62bb141896c4119"}
Sep 06 04:55:42.443174 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.443136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9cb22" event={"ID":"9122d1d8-9c3f-412c-9350-f380c015475b","Type":"ContainerStarted","Data":"f395d0a57e8468529d04900d34ab1b717b66de66e0b05b7aaf0486953abbb729"}
Sep 06 04:55:42.447794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.447759    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rgc5h" event={"ID":"a452e6b1-85c2-485d-8d50-3434e01caae4","Type":"ContainerStarted","Data":"a771d369500b9e14fa75b5010faf8b68ca9c5aa4e3219410c4502b06976bd392"}
Sep 06 04:55:42.457651 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.457613    2576 generic.go:356] "Generic (PLEG): container finished" podID="1dff361b9784023f163739c4d30ad35d" containerID="e179c5d8137e3a9ec02c5de4113f3033975fc81d60caeec6cd9a9fdf4fd88958" exitCode=0
Sep 06 04:55:42.457806 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.457715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"1dff361b9784023f163739c4d30ad35d","Type":"ContainerDied","Data":"e179c5d8137e3a9ec02c5de4113f3033975fc81d60caeec6cd9a9fdf4fd88958"}
Sep 06 04:55:42.471776 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.471742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkzgd" event={"ID":"91727639-16b7-49f3-9277-5a6806eb3237","Type":"ContainerStarted","Data":"5f61287e01865af49df355e8c8aace19769a05fcc5d6ec0ccae30591340a29d8"}
Sep 06 04:55:42.484728 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.484645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr" event={"ID":"51ee04e0-3c86-4c9a-902e-e9040d8fa344","Type":"ContainerStarted","Data":"28da12b3a00a174d6d32b36908eecbad13f6bf24208178cb87a2e1c14204a8ba"}
Sep 06 04:55:42.491613 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.491532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7k4dl" event={"ID":"7909525c-4e27-41e3-881f-8f7127d0b124","Type":"ContainerStarted","Data":"27d1bcb4c58e014b88a5be549eafabd27ccb2fa72f2f24326bcde7fa1131a258"}
Sep 06 04:55:42.522792 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.522692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerStarted","Data":"6fcb149aa69893d0e0566e0f3abe91fc8b7e6939b0bb71129a320d47d13b1700"}
Sep 06 04:55:42.534168 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:42.534130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"89ee4de97c9db204e1ce27b68da57bce8168ac2351c2a1e67d5c5329f7a5f805"}
Sep 06 04:55:43.381106 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:43.380264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:43.381106 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:43.380684    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:43.558109 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:43.558033    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" event={"ID":"1dff361b9784023f163739c4d30ad35d","Type":"ContainerStarted","Data":"59b396d9a05bb860429a5196320aa6f5cc55dd2241da04e427a382ab8440a0f3"}
Sep 06 04:55:43.952691 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:43.952645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:43.952939 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:43.952820    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:43.952939 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:43.952844    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:43.952939 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:43.952857    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:43.952939 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:43.952922    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:47.952901403 +0000 UTC m=+10.041979091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:44.054128 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:44.054090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:44.054128 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:44.054155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:44.054479 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.054318    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:44.054479 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.054387    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:48.054367323 +0000 UTC m=+10.143445000 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:44.054827 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.054800    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:44.054902 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.054857    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:55:48.054841589 +0000 UTC m=+10.143919270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:44.380967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:44.380857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:44.380967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:44.380944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:44.381570 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.381029    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:44.381570 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:44.381165    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:45.380355 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:45.379804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:45.380355 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:45.379971    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:46.382139 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:46.382107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:46.382641 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:46.382107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:46.382641 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:46.382295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:46.382641 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:46.382487    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:47.380006 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:47.379899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:47.380309 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:47.380116    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:47.986286 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:47.986187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:47.986834 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:47.986377    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:47.986834 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:47.986401    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:47.986834 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:47.986415    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:47.986834 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:47.986489    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:55.986466029 +0000 UTC m=+18.075543716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:48.086953 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:48.086897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:48.087156 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:48.087000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:48.087156 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.087095    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:48.087156 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.087128    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:48.087321 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.087172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:55:56.087149903 +0000 UTC m=+18.176227576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:48.087321 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.087202    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:55:56.087179394 +0000 UTC m=+18.176257088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:48.385088 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:48.384580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:48.385088 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.384664    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:48.385088 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:48.384842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:48.385088 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:48.385004    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:49.379861 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:49.379806    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:49.380324 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:49.379981    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:50.381052 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:50.381016    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:50.381511 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:50.381151    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:50.382008 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:50.381985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:50.382143 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:50.382101    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:51.379646 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:51.379615    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:51.379827 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:51.379725    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:52.380053 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:52.380021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:52.380053 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:52.380052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:52.380538 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:52.380143    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:52.380538 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:52.380285    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:53.380105 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:53.380052    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:53.380497 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:53.380214    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:54.380001 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:54.379963    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:54.380200 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:54.379959    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:54.380200 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:54.380114    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:54.380200 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:54.380182    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:55.379432 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:55.379397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:55.379610 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:55.379538    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:56.042965 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:56.042926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:56.043371 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.043093    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:55:56.043371 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.043118    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:55:56.043371 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.043132    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:56.043371 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.043186    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:12.043172509 +0000 UTC m=+34.132250189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:55:56.144176 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:56.144133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:56.144357 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:56.144240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:56.144357 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.144286    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:56.144357 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.144354    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:56.144483 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.144362    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:12.144341188 +0000 UTC m=+34.233418876 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:55:56.144483 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.144405    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:56:12.144387939 +0000 UTC m=+34.233465613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:55:56.380250 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:56.380163    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:56.380398 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.380305    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:56.380398 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:56.380363    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:56.380540 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:56.380503    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:57.380067 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:57.380034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:57.380521 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:57.380181    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:55:58.380872 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:58.380838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:55:58.381326 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:58.380964    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:55:58.381326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:58.381015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:55:58.381326 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:58.381122    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:55:59.380560 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:55:59.380100    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:55:59.380560 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:55:59.380244    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:00.379924 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.379697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:00.380404 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.379749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:00.380404 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:00.380014    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:00.380404 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:00.380114    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:00.594183 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.594153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7k4dl" event={"ID":"7909525c-4e27-41e3-881f-8f7127d0b124","Type":"ContainerStarted","Data":"a40988c23dcc5ab8be6662184c8fffb41ebfb304d17b93b964c069843858a191"}
Sep 06 04:56:00.595730 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.595703    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="426f52ec97db6674a92cf1932fcefba13c49047b731586f8e08e3873d4e89403" exitCode=0
Sep 06 04:56:00.595848 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.595788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"426f52ec97db6674a92cf1932fcefba13c49047b731586f8e08e3873d4e89403"}
Sep 06 04:56:00.599045 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599027    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 04:56:00.599416 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599393    2576 generic.go:356] "Generic (PLEG): container finished" podID="d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c" containerID="2057cba91bac96dc2cd76cfb69fd4ade2a6c14328906152255100bb6b613da57" exitCode=1
Sep 06 04:56:00.599507 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"f1e7f1e7996958191441c97241ac5ea50cd82cb8d71d093dd0d2277f659f2238"}
Sep 06 04:56:00.599559 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"715f4e934182265414d79b1425b6d72814983b0598ab90c5dcf920fc1e780d6d"}
Sep 06 04:56:00.599607 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"32846ffd300471e3a6fee0e262386efc10723919c7dc1d5480ab9ece3581436f"}
Sep 06 04:56:00.599671 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerDied","Data":"2057cba91bac96dc2cd76cfb69fd4ade2a6c14328906152255100bb6b613da57"}
Sep 06 04:56:00.599710 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.599698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"ea652453347dbdbdddd79e87366f7c4dcc5baa52d0b2a1e536d9b65d63edbb20"}
Sep 06 04:56:00.601114 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.601093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-plp8r" event={"ID":"4d7bc33c-3588-4f47-8356-5be260b42ec6","Type":"ContainerStarted","Data":"5ce2b7f8bf8bc706603a2add3ff26936fb0d5d4ebfac003b453cf1a90d8b2e76"}
Sep 06 04:56:00.602433 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.602404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jwsh9" event={"ID":"544f66cd-a01a-480b-aea0-23c82fc314c0","Type":"ContainerStarted","Data":"7ea911cc2cc42668aeb6a32bb12aa76c0b0841a0a475daefff1e9caaaecbdd03"}
Sep 06 04:56:00.603817 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.603796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rgc5h" event={"ID":"a452e6b1-85c2-485d-8d50-3434e01caae4","Type":"ContainerStarted","Data":"873b42a4ab1ab5d9df9e35edecd1256edec4b31836a1cb741faa388ccb83dd22"}
Sep 06 04:56:00.605459 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.605436    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkzgd" event={"ID":"91727639-16b7-49f3-9277-5a6806eb3237","Type":"ContainerStarted","Data":"a1051c9c4744ccfab5231a2bd524878cd8fc39bffcaeb8a0a827ccfd4eb20cba"}
Sep 06 04:56:00.607167 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.607146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr" event={"ID":"51ee04e0-3c86-4c9a-902e-e9040d8fa344","Type":"ContainerStarted","Data":"ed32ab359e799b7e14ff9bc337abc0726c415752c8d0264291362c81f2a35301"}
Sep 06 04:56:00.615955 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.615877    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-133.ec2.internal" podStartSLOduration=20.615860391 podStartE2EDuration="20.615860391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:55:43.575123545 +0000 UTC m=+5.664201243" watchObservedRunningTime="2026-09-06 04:56:00.615860391 +0000 UTC m=+22.704938138"
Sep 06 04:56:00.647621 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.647579    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7k4dl" podStartSLOduration=3.174065455 podStartE2EDuration="21.647564151s" totalImagesPullingTime="18.473498696s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:00.647116052 +0000 UTC m=+22.736193745" watchObservedRunningTime="2026-09-06 04:56:00.647564151 +0000 UTC m=+22.736641843"
Sep 06 04:56:00.664755 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.664709    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jwsh9" podStartSLOduration=3.252667713 podStartE2EDuration="21.664694757s" totalImagesPullingTime="18.412027044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:00.6640867 +0000 UTC m=+22.753164389" watchObservedRunningTime="2026-09-06 04:56:00.664694757 +0000 UTC m=+22.753772451"
Sep 06 04:56:00.690708 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.690511    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rgc5h" podStartSLOduration=3.2696959469999998 podStartE2EDuration="21.690490433s" totalImagesPullingTime="18.420794486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:00.677966333 +0000 UTC m=+22.767044025" watchObservedRunningTime="2026-09-06 04:56:00.690490433 +0000 UTC m=+22.779568123"
Sep 06 04:56:00.691113 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.691082    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rkzgd" podStartSLOduration=10.985539197 podStartE2EDuration="21.691072055s" totalImagesPullingTime="10.705532858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:00.690775558 +0000 UTC m=+22.779853233" watchObservedRunningTime="2026-09-06 04:56:00.691072055 +0000 UTC m=+22.780149747"
Sep 06 04:56:00.711434 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:00.711375    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-plp8r" podStartSLOduration=3.304729261 podStartE2EDuration="21.711355822s" totalImagesPullingTime="18.406626561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:00.710523877 +0000 UTC m=+22.799601570" watchObservedRunningTime="2026-09-06 04:56:00.711355822 +0000 UTC m=+22.800433516"
Sep 06 04:56:01.119327 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.119232    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 06 04:56:01.379799 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.379711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:01.379980 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:01.379850    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:01.380743 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.380613    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-06T04:56:01.119260745Z","UUID":"b3e516fa-c719-46c7-be60-7d94c2294a99","Handler":null,"Name":"","Endpoint":""}
Sep 06 04:56:01.382151 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.382134    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 06 04:56:01.382241 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.382164    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 06 04:56:01.612162 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.612086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9cb22" event={"ID":"9122d1d8-9c3f-412c-9350-f380c015475b","Type":"ContainerStarted","Data":"8d822279b03cd35ef37dd6071d66c56f030aec1455c0729ee0ed22928d012574"}
Sep 06 04:56:01.614632 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.614602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr" event={"ID":"51ee04e0-3c86-4c9a-902e-e9040d8fa344","Type":"ContainerStarted","Data":"097d10e854eae3192c135cc3219f7e07d94ced70647fbe849fa5878f1a7f21a0"}
Sep 06 04:56:01.618488 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.618457    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 04:56:01.619006 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.618982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"f289ce3efb174db5a5b75ecdab2188653b913c423ab89f7d9e14aae0a35919a5"}
Sep 06 04:56:01.626698 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:01.626659    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9cb22" podStartSLOduration=4.211751737 podStartE2EDuration="22.626645756s" totalImagesPullingTime="18.414894019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:01.625849772 +0000 UTC m=+23.714927465" watchObservedRunningTime="2026-09-06 04:56:01.626645756 +0000 UTC m=+23.715723448"
Sep 06 04:56:02.379813 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:02.379775    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:02.379989 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:02.379775    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:02.379989 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:02.379923    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:02.380377 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:02.379989    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:02.624228 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:02.624127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr" event={"ID":"51ee04e0-3c86-4c9a-902e-e9040d8fa344","Type":"ContainerStarted","Data":"cb5c647ee2411d079099cf97ad5a5ff58c2c18319d359987dd6de0f345136450"}
Sep 06 04:56:02.641326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:02.641272    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l28nr" podStartSLOduration=3.261014735 podStartE2EDuration="23.641256245s" totalImagesPullingTime="20.38024151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:02.640859397 +0000 UTC m=+24.729937091" watchObservedRunningTime="2026-09-06 04:56:02.641256245 +0000 UTC m=+24.730333946"
Sep 06 04:56:03.379873 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:03.379836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:03.380046 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:03.379980    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:03.630287 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:03.630209    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 04:56:03.630649 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:03.630613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"daf5da9c1122e62da3d5d8a2d77b169d254abfc85581944f8f7a5e61383fcf30"}
Sep 06 04:56:04.317943 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.317886    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:56:04.318521 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.318494    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:56:04.379339 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.379309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:04.379521 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.379309    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:04.379521 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:04.379445    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:04.379614 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:04.379518    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:04.632714 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.632642    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:56:04.633219 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:04.633186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rkzgd"
Sep 06 04:56:05.379268 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.379221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:05.379465 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:05.379338    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:05.637014 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.636937    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="76a65488e25f990faeb8ede40d628a91e5a4b3f73536699ab334a4d84a138e06" exitCode=0
Sep 06 04:56:05.637726 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.637013    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"76a65488e25f990faeb8ede40d628a91e5a4b3f73536699ab334a4d84a138e06"}
Sep 06 04:56:05.640749 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.640732    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 04:56:05.641195 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.641164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"66138671aad47b0267ba471829081132136abbc116f005eb8320860117edc4fd"}
Sep 06 04:56:05.644686 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.644668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:05.644775 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.644723    2576 scope.go:117] "RemoveContainer" containerID="2057cba91bac96dc2cd76cfb69fd4ade2a6c14328906152255100bb6b613da57"
Sep 06 04:56:05.661509 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:05.661486    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:06.380207 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.380177    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:06.380311 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.380222    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:06.380382 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:06.380313    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:06.380472 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:06.380453    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:06.645630 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.645447    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="ed5da732872ae38be6c62086030ca1046c32ee674414ff88f138cdf8b4b2a3a9" exitCode=0
Sep 06 04:56:06.646152 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.645527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"ed5da732872ae38be6c62086030ca1046c32ee674414ff88f138cdf8b4b2a3a9"}
Sep 06 04:56:06.650067 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.650025    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 04:56:06.650435 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.650410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" event={"ID":"d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c","Type":"ContainerStarted","Data":"4e8b7db0770ffd87d9bcce712df6b7648775f0ccefd933b7302ab538bb1f917d"}
Sep 06 04:56:06.650507 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.650483    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 06 04:56:06.650647 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.650630    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:06.666610 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.666587    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:06.711353 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.711303    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb" podStartSLOduration=9.258714041 podStartE2EDuration="27.711287805s" totalImagesPullingTime="18.452573764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:06.710963216 +0000 UTC m=+28.800040909" watchObservedRunningTime="2026-09-06 04:56:06.711287805 +0000 UTC m=+28.800365498"
Sep 06 04:56:06.800345 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.800306    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vxsx"]
Sep 06 04:56:06.800520 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.800443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:06.800585 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:06.800560    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:06.803267 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.803239    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7t92"]
Sep 06 04:56:06.803400 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.803353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:06.803498 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:06.803475    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:06.803893 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.803870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nfv2z"]
Sep 06 04:56:06.803991 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.803961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:06.804063 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:06.804043    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:06.895563 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:06.895530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:07.655314 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:07.655234    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="174da40f81a958c110d659dfc0356f90f500255216acd8b66c4e6e562af2244b" exitCode=0
Sep 06 04:56:07.655638 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:07.655319    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"174da40f81a958c110d659dfc0356f90f500255216acd8b66c4e6e562af2244b"}
Sep 06 04:56:08.382171 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:08.382137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:08.382337 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:08.382137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:08.382337 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:08.382242    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:08.382337 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:08.382315    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:08.382337 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:08.382137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:08.382553 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:08.382398    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:10.383649 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:10.383616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:10.384100 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:10.383731    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:10.384100 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:10.383846    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:10.384100 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:10.383913    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:10.384237 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:10.384093    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:10.384237 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:10.384134    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:12.060055 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.060012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:12.060536 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.060202    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 06 04:56:12.060536 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.060225    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 06 04:56:12.060536 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.060235    2576 projected.go:196] Error preparing data for projected volume kube-api-access-v6wdn for pod openshift-network-diagnostics/network-check-target-nfv2z: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:56:12.060536 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.060289    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn podName:672860b2-6967-44fd-ac6d-4523fbca1843 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:44.060273955 +0000 UTC m=+66.149351625 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6wdn" (UniqueName: "kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn") pod "network-check-target-nfv2z" (UID: "672860b2-6967-44fd-ac6d-4523fbca1843") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 06 04:56:12.160693 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.160651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:12.160875 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.160725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:12.160875 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.160833    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:56:12.160994 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.160837    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 06 04:56:12.160994 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.160937    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:56:44.160918425 +0000 UTC m=+66.249996095 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 06 04:56:12.160994 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.160973    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret podName:9e3acc0d-6ef3-4709-a29a-79e9de3b703e nodeName:}" failed. No retries permitted until 2026-09-06 04:56:44.160958434 +0000 UTC m=+66.250036104 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret") pod "global-pull-secret-syncer-8vxsx" (UID: "9e3acc0d-6ef3-4709-a29a-79e9de3b703e") : object "kube-system"/"original-pull-secret" not registered
Sep 06 04:56:12.382492 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.382422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:12.382643 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.382422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:12.382643 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.382525    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8vxsx" podUID="9e3acc0d-6ef3-4709-a29a-79e9de3b703e"
Sep 06 04:56:12.382643 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.382422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:12.382643 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.382604    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:56:12.382769 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.382711    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfv2z" podUID="672860b2-6967-44fd-ac6d-4523fbca1843"
Sep 06 04:56:12.710128 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.710106    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-133.ec2.internal" event="NodeReady"
Sep 06 04:56:12.710245 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.710237    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 06 04:56:12.752300 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.752266    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v47hr"]
Sep 06 04:56:12.779623 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.779589    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6hzl5"]
Sep 06 04:56:12.779747 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.779724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.782555 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.782531    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 06 04:56:12.782701 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.782534    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-prh2m\""
Sep 06 04:56:12.782701 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.782574    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 06 04:56:12.797517 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.797489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hzl5"]
Sep 06 04:56:12.797658 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.797527    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v47hr"]
Sep 06 04:56:12.797658 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.797603    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.800098 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.800074    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 06 04:56:12.800293 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.800110    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7h5qb\""
Sep 06 04:56:12.800293 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.800118    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 06 04:56:12.800407 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.800303    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 06 04:56:12.865876 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.865672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-tmp-dir\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.865876 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.865879    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmhr7\" (UniqueName: \"kubernetes.io/projected/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-kube-api-access-zmhr7\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.866080 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.865919    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-config-volume\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.866080 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.865970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-cert\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.866080 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.865985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96jrk\" (UniqueName: \"kubernetes.io/projected/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-kube-api-access-96jrk\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.866080 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.866015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.966548 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-cert\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.966548 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96jrk\" (UniqueName: \"kubernetes.io/projected/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-kube-api-access-96jrk\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.966548 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.966817 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-tmp-dir\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.966817 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmhr7\" (UniqueName: \"kubernetes.io/projected/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-kube-api-access-zmhr7\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.966817 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-config-volume\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.966817 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.966700    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:12.966817 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:12.966773    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:13.466751774 +0000 UTC m=+35.555829462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:12.967084 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.966993    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-tmp-dir\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.967326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.967307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-config-volume\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.970922 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.970885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-cert\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:12.976592 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.976073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmhr7\" (UniqueName: \"kubernetes.io/projected/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-kube-api-access-zmhr7\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:12.976592 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:12.976298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96jrk\" (UniqueName: \"kubernetes.io/projected/80a2d8d2-7870-4e16-aaee-c242d7ea3c0c-kube-api-access-96jrk\") pod \"ingress-canary-6hzl5\" (UID: \"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c\") " pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:13.105605 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.105563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hzl5"
Sep 06 04:56:13.415180 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.415108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hzl5"]
Sep 06 04:56:13.418270 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:56:13.418232    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a2d8d2_7870_4e16_aaee_c242d7ea3c0c.slice/crio-6a9012f2db233dd88735b80ad3be65411368b1665e864e1254aa1fb77179a83d WatchSource:0}: Error finding container 6a9012f2db233dd88735b80ad3be65411368b1665e864e1254aa1fb77179a83d: Status 404 returned error can't find the container with id 6a9012f2db233dd88735b80ad3be65411368b1665e864e1254aa1fb77179a83d
Sep 06 04:56:13.470398 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.470361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:13.470539 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:13.470471    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:13.470539 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:13.470529    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:14.470510973 +0000 UTC m=+36.559588643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:13.672284 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.672255    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="74d12d6421edc636e6bfa6dbd38b2f77e50e9e9bc1ac209d19c8b366c8967ab3" exitCode=0
Sep 06 04:56:13.672452 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.672336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"74d12d6421edc636e6bfa6dbd38b2f77e50e9e9bc1ac209d19c8b366c8967ab3"}
Sep 06 04:56:13.673534 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:13.673510    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hzl5" event={"ID":"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c","Type":"ContainerStarted","Data":"6a9012f2db233dd88735b80ad3be65411368b1665e864e1254aa1fb77179a83d"}
Sep 06 04:56:14.379680 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.379643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:14.379680 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.379683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:14.380659 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.379643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:14.382341 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.382316    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b9kvk\""
Sep 06 04:56:14.383836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.383322    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 04:56:14.383836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.383389    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 04:56:14.383836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.383321    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzjkx\""
Sep 06 04:56:14.383836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.383639    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 04:56:14.383836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.383688    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 04:56:14.478252 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.478207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:14.478420 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:14.478337    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:14.478420 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:14.478399    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:16.478378711 +0000 UTC m=+38.567456643 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:14.680058 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.680027    2576 generic.go:356] "Generic (PLEG): container finished" podID="2757f210-9c9b-4762-8631-9348f13dc03e" containerID="f7c28ed997bdf7e405e437218b8761dfdb280cbde3962ee338730ba1bf6b170f" exitCode=0
Sep 06 04:56:14.680229 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:14.680106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerDied","Data":"f7c28ed997bdf7e405e437218b8761dfdb280cbde3962ee338730ba1bf6b170f"}
Sep 06 04:56:15.686273 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:15.686084    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" event={"ID":"2757f210-9c9b-4762-8631-9348f13dc03e","Type":"ContainerStarted","Data":"9481f9fad50a66abe8062ef36508328431aa2bb4f8c582fad6c1da0207ab97e3"}
Sep 06 04:56:15.706710 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:15.706658    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pjrd9" podStartSLOduration=7.8213193709999995 podStartE2EDuration="36.706640965s" totalImagesPullingTime="28.885321594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:15.705198922 +0000 UTC m=+37.794276614" watchObservedRunningTime="2026-09-06 04:56:15.706640965 +0000 UTC m=+37.795718657"
Sep 06 04:56:16.488254 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:16.488220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:16.488427 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:16.488370    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:16.488505 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:16.488435    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:20.488418535 +0000 UTC m=+42.577496225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:16.690753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:16.690711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hzl5" event={"ID":"80a2d8d2-7870-4e16-aaee-c242d7ea3c0c","Type":"ContainerStarted","Data":"f8a4bbf5595d146e95855e6e669f0a89b5cd0e05a3ef02505d389ee1295122ba"}
Sep 06 04:56:16.705790 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:16.705736    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6hzl5" podStartSLOduration=2.03590175 podStartE2EDuration="4.705721114s" totalImagesPullingTime="2.669819364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:16.70444128 +0000 UTC m=+38.793518975" watchObservedRunningTime="2026-09-06 04:56:16.705721114 +0000 UTC m=+38.794798822"
Sep 06 04:56:20.515507 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:20.515463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:20.515938 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:20.515617    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:20.515938 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:20.515683    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:28.515666317 +0000 UTC m=+50.604744010 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:28.573169 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:28.573118    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:28.573634 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:28.573281    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:28.573634 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:28.573351    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:56:44.573333294 +0000 UTC m=+66.662410992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:38.669365 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:38.669333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pmmpb"
Sep 06 04:56:44.077423 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.077389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:44.080839 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.080821    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 06 04:56:44.089967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.089947    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 06 04:56:44.102668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.102648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wdn\" (UniqueName: \"kubernetes.io/projected/672860b2-6967-44fd-ac6d-4523fbca1843-kube-api-access-v6wdn\") pod \"network-check-target-nfv2z\" (UID: \"672860b2-6967-44fd-ac6d-4523fbca1843\") " pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:44.178744 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.178700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:56:44.178969 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.178787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:44.181582 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.181563    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 06 04:56:44.181791 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.181776    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 06 04:56:44.189797 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:44.189780    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:56:44.189875 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:44.189864    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:57:48.189841718 +0000 UTC m=+130.278919395 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : secret "metrics-daemon-secret" not found
Sep 06 04:56:44.192068 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.192049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9e3acc0d-6ef3-4709-a29a-79e9de3b703e-original-pull-secret\") pod \"global-pull-secret-syncer-8vxsx\" (UID: \"9e3acc0d-6ef3-4709-a29a-79e9de3b703e\") " pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:44.404700 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.402259    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dzjkx\""
Sep 06 04:56:44.405758 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.405739    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8vxsx"
Sep 06 04:56:44.409962 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.409942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:44.582395 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.582348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:56:44.582564 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:44.582498    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:56:44.582564 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:56:44.582562    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:57:16.582544657 +0000 UTC m=+98.671622327 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:56:44.701015 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.700984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8vxsx"]
Sep 06 04:56:44.704317 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:56:44.704286    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3acc0d_6ef3_4709_a29a_79e9de3b703e.slice/crio-0f71ef6de424f02a5febb947769bf2b696477796172bf42c99ec00ed7998e171 WatchSource:0}: Error finding container 0f71ef6de424f02a5febb947769bf2b696477796172bf42c99ec00ed7998e171: Status 404 returned error can't find the container with id 0f71ef6de424f02a5febb947769bf2b696477796172bf42c99ec00ed7998e171
Sep 06 04:56:44.727726 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.727696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nfv2z"]
Sep 06 04:56:44.730689 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:56:44.730664    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672860b2_6967_44fd_ac6d_4523fbca1843.slice/crio-46461da38aefb59724a292cbc7e7c6a2aa057c822bfd3bdc39b171e86b2aaf56 WatchSource:0}: Error finding container 46461da38aefb59724a292cbc7e7c6a2aa057c822bfd3bdc39b171e86b2aaf56: Status 404 returned error can't find the container with id 46461da38aefb59724a292cbc7e7c6a2aa057c822bfd3bdc39b171e86b2aaf56
Sep 06 04:56:44.757808 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.757769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nfv2z" event={"ID":"672860b2-6967-44fd-ac6d-4523fbca1843","Type":"ContainerStarted","Data":"46461da38aefb59724a292cbc7e7c6a2aa057c822bfd3bdc39b171e86b2aaf56"}
Sep 06 04:56:44.758879 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:44.758859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vxsx" event={"ID":"9e3acc0d-6ef3-4709-a29a-79e9de3b703e","Type":"ContainerStarted","Data":"0f71ef6de424f02a5febb947769bf2b696477796172bf42c99ec00ed7998e171"}
Sep 06 04:56:49.773825 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:49.773785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8vxsx" event={"ID":"9e3acc0d-6ef3-4709-a29a-79e9de3b703e","Type":"ContainerStarted","Data":"f5e99fffe33d52f8ca9885899204232f8789844514f7194a00aeef8e5fabe176"}
Sep 06 04:56:49.775216 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:49.775190    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nfv2z" event={"ID":"672860b2-6967-44fd-ac6d-4523fbca1843","Type":"ContainerStarted","Data":"a5c3543832c647ff5e607f75ac5ac9e2b25f588f44dcf4676f0e8d01c54c7c6c"}
Sep 06 04:56:49.775332 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:49.775314    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:56:49.787687 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:49.787634    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8vxsx" podStartSLOduration=66.373423938 podStartE2EDuration="1m10.78762129s" totalImagesPullingTime="4.414197352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:49.787456721 +0000 UTC m=+71.876534415" watchObservedRunningTime="2026-09-06 04:56:49.78762129 +0000 UTC m=+71.876698983"
Sep 06 04:56:49.800802 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:49.800758    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nfv2z" podStartSLOduration=66.416098204 podStartE2EDuration="1m10.800747513s" totalImagesPullingTime="4.384649309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:56:49.799852211 +0000 UTC m=+71.888929896" watchObservedRunningTime="2026-09-06 04:56:49.800747513 +0000 UTC m=+71.889825205"
Sep 06 04:56:59.797969 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.797832    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"]
Sep 06 04:56:59.800593 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.800575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.802993 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.802972    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 06 04:56:59.804101 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.804081    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 06 04:56:59.804198 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.804104    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 06 04:56:59.804384 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.804368    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 06 04:56:59.808424 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.808403    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"]
Sep 06 04:56:59.889778 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.889736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad7409b5-2e1f-4610-8a79-c310b21ff457-tmp\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.889778 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.889777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad7409b5-2e1f-4610-8a79-c310b21ff457-klusterlet-config\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.890032 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.889798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8t9b\" (UniqueName: \"kubernetes.io/projected/ad7409b5-2e1f-4610-8a79-c310b21ff457-kube-api-access-b8t9b\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.990469 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.990418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad7409b5-2e1f-4610-8a79-c310b21ff457-tmp\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.990469 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.990470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad7409b5-2e1f-4610-8a79-c310b21ff457-klusterlet-config\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.990677 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.990493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8t9b\" (UniqueName: \"kubernetes.io/projected/ad7409b5-2e1f-4610-8a79-c310b21ff457-kube-api-access-b8t9b\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.990801 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.990776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad7409b5-2e1f-4610-8a79-c310b21ff457-tmp\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.993185 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.993157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ad7409b5-2e1f-4610-8a79-c310b21ff457-klusterlet-config\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:56:59.998261 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:56:59.998242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8t9b\" (UniqueName: \"kubernetes.io/projected/ad7409b5-2e1f-4610-8a79-c310b21ff457-kube-api-access-b8t9b\") pod \"klusterlet-addon-workmgr-55dd9947d5-npsxm\" (UID: \"ad7409b5-2e1f-4610-8a79-c310b21ff457\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:57:00.110297 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:00.110203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:57:00.378767 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:00.378677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"]
Sep 06 04:57:00.383091 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:57:00.383062    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7409b5_2e1f_4610_8a79_c310b21ff457.slice/crio-ad83fb604712052bfa4d93b86693d7adf0e67db687cdafc7d9c68359483b375d WatchSource:0}: Error finding container ad83fb604712052bfa4d93b86693d7adf0e67db687cdafc7d9c68359483b375d: Status 404 returned error can't find the container with id ad83fb604712052bfa4d93b86693d7adf0e67db687cdafc7d9c68359483b375d
Sep 06 04:57:00.803158 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:00.803120    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm" event={"ID":"ad7409b5-2e1f-4610-8a79-c310b21ff457","Type":"ContainerStarted","Data":"ad83fb604712052bfa4d93b86693d7adf0e67db687cdafc7d9c68359483b375d"}
Sep 06 04:57:04.815836 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:04.815799    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm" event={"ID":"ad7409b5-2e1f-4610-8a79-c310b21ff457","Type":"ContainerStarted","Data":"5abde902b5de10927eb4c2c42141aeb849f36763f4d3de75e46f7a4b028948ec"}
Sep 06 04:57:04.816266 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:04.815972    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:57:04.817696 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:04.817673    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:57:04.830370 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:04.830333    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm" podStartSLOduration=2.274208155 podStartE2EDuration="5.830320681s" totalImagesPullingTime="3.556112526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:57:04.82986854 +0000 UTC m=+86.918946233" watchObservedRunningTime="2026-09-06 04:57:04.830320681 +0000 UTC m=+86.919398430"
Sep 06 04:57:16.605521 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:16.605429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:57:16.605868 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:57:16.605550    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:57:16.605868 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:57:16.605632    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 04:58:20.605617648 +0000 UTC m=+162.694695333 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:57:20.779782 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:20.779748    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nfv2z"
Sep 06 04:57:48.223220 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:57:48.223164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:57:48.223701 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:57:48.223316    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:57:48.223701 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:57:48.223393    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:50.223377028 +0000 UTC m=+252.312454699 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : secret "metrics-daemon-secret" not found
Sep 06 04:58:15.787831 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:58:15.787781    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-v47hr" podUID="12b770a7-9c8d-4f77-8d3a-0b8f98ad91db"
Sep 06 04:58:15.991500 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:58:15.991472    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v47hr"
Sep 06 04:58:17.391966 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:58:17.391921    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 04:58:20.646955 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:58:20.646877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 04:58:20.647641 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:58:20.647060    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 06 04:58:20.647641 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:58:20.647155    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls podName:12b770a7-9c8d-4f77-8d3a-0b8f98ad91db nodeName:}" failed. No retries permitted until 2026-09-06 05:00:22.64713143 +0000 UTC m=+284.736209102 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls") pod "dns-default-v47hr" (UID: "12b770a7-9c8d-4f77-8d3a-0b8f98ad91db") : secret "dns-default-metrics-tls" not found
Sep 06 04:58:28.380645 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:58:28.380600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:59:04.103584 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:04.103553    2576 generic.go:356] "Generic (PLEG): container finished" podID="ad7409b5-2e1f-4610-8a79-c310b21ff457" containerID="5abde902b5de10927eb4c2c42141aeb849f36763f4d3de75e46f7a4b028948ec" exitCode=1
Sep 06 04:59:04.104041 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:04.103614    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm" event={"ID":"ad7409b5-2e1f-4610-8a79-c310b21ff457","Type":"ContainerDied","Data":"5abde902b5de10927eb4c2c42141aeb849f36763f4d3de75e46f7a4b028948ec"}
Sep 06 04:59:04.104041 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:04.103959    2576 scope.go:117] "RemoveContainer" containerID="5abde902b5de10927eb4c2c42141aeb849f36763f4d3de75e46f7a4b028948ec"
Sep 06 04:59:04.816306 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:04.816253    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:59:05.109000 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:05.108891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm" event={"ID":"ad7409b5-2e1f-4610-8a79-c310b21ff457","Type":"ContainerStarted","Data":"a86d4e4ed19b3388bbef00183f3bc42e5860b8cfe1d45b6e44e4eef3f641c1db"}
Sep 06 04:59:05.109435 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:05.109102    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:59:05.109671 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:05.109654    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55dd9947d5-npsxm"
Sep 06 04:59:41.382109 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.382070    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"]
Sep 06 04:59:41.385103 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.385083    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"]
Sep 06 04:59:41.385219 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.385203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"
Sep 06 04:59:41.387704 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.387675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h44g8\""
Sep 06 04:59:41.387822 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.387716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.388039 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.388015    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 06 04:59:41.388142 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.388129    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:59:41.389880 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.389864    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 06 04:59:41.391962 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.391941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 06 04:59:41.392061 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.391977    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 06 04:59:41.392061 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.392001    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 06 04:59:41.392165 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.391983    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-n6c9p\""
Sep 06 04:59:41.395615 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.395596    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"]
Sep 06 04:59:41.396517 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.396496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6mzs\" (UniqueName: \"kubernetes.io/projected/c580bd12-8c7c-434a-8a45-a003ca171311-kube-api-access-h6mzs\") pod \"volume-data-source-validator-695c895bd6-gntt5\" (UID: \"c580bd12-8c7c-434a-8a45-a003ca171311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"
Sep 06 04:59:41.399844 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.399821    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"]
Sep 06 04:59:41.483150 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.483122    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t55rq"]
Sep 06 04:59:41.485965 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.485946    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjfnp"]
Sep 06 04:59:41.486091 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.486073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.488788 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.488769    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 06 04:59:41.488885 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.488861    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 06 04:59:41.489098 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.489087    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 06 04:59:41.489148 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.489108    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 06 04:59:41.489339 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.489321    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hcdh6\""
Sep 06 04:59:41.489417 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.489349    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57b77cb778-p6mrm"]
Sep 06 04:59:41.489546 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.489530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.492119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492091    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 04:59:41.492198 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.492583 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492518    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 06 04:59:41.492583 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492523    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 06 04:59:41.492778 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492750    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qrkz4\""
Sep 06 04:59:41.492880 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 06 04:59:41.492880 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.492871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 06 04:59:41.495119 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.495101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.496373 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.495998    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 06 04:59:41.496373 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.496221    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g4kss\""
Sep 06 04:59:41.496531 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.496476    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 06 04:59:41.496839 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.496817    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-config\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.496994 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.496975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h4jb\" (UniqueName: \"kubernetes.io/projected/45d3705d-364e-4846-b3d0-e10876db24d1-kube-api-access-4h4jb\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.497066 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.497066 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45d3705d-364e-4846-b3d0-e10876db24d1-serving-cert\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.497173 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lf8\" (UniqueName: \"kubernetes.io/projected/109e5587-86c1-487f-8166-f8e29284cfdf-kube-api-access-25lf8\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.497173 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-trusted-ca\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.497173 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/109e5587-86c1-487f-8166-f8e29284cfdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.497314 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mzs\" (UniqueName: \"kubernetes.io/projected/c580bd12-8c7c-434a-8a45-a003ca171311-kube-api-access-h6mzs\") pod \"volume-data-source-validator-695c895bd6-gntt5\" (UID: \"c580bd12-8c7c-434a-8a45-a003ca171311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"
Sep 06 04:59:41.497314 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497179    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 06 04:59:41.497569 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497547    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 06 04:59:41.497662 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497592    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 06 04:59:41.497662 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497648    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 06 04:59:41.497797 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.497776    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 06 04:59:41.498526 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.498508    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 06 04:59:41.498619 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.498526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 06 04:59:41.498701 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.498636    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fgl4q\""
Sep 06 04:59:41.498967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.498777    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 06 04:59:41.498967 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.498879    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t55rq"]
Sep 06 04:59:41.499276 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.499257    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 06 04:59:41.500025 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.499969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjfnp"]
Sep 06 04:59:41.502163 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.502142    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57b77cb778-p6mrm"]
Sep 06 04:59:41.503295 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.503275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 04:59:41.504936 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.504917    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 06 04:59:41.513864 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.513841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mzs\" (UniqueName: \"kubernetes.io/projected/c580bd12-8c7c-434a-8a45-a003ca171311-kube-api-access-h6mzs\") pod \"volume-data-source-validator-695c895bd6-gntt5\" (UID: \"c580bd12-8c7c-434a-8a45-a003ca171311\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"
Sep 06 04:59:41.598423 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.598423 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-stats-auth\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598633    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.598753 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lf8\" (UniqueName: \"kubernetes.io/projected/109e5587-86c1-487f-8166-f8e29284cfdf-kube-api-access-25lf8\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5db9\" (UniqueName: \"kubernetes.io/projected/7062649b-61ff-45c2-8c19-3277489659bc-kube-api-access-w5db9\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.598824    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.598930    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls podName:109e5587-86c1-487f-8166-f8e29284cfdf nodeName:}" failed. No retries permitted until 2026-09-06 04:59:42.098891685 +0000 UTC m=+244.187969356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-kpp95" (UID: "109e5587-86c1-487f-8166-f8e29284cfdf") : secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzwjp\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.598988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-default-certificate\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-config\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-snapshots\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c23ef83-d706-4e78-92c7-2da117488b31-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t5xm\" (UniqueName: \"kubernetes.io/projected/1c23ef83-d706-4e78-92c7-2da117488b31-kube-api-access-5t5xm\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.599131 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h4jb\" (UniqueName: \"kubernetes.io/projected/45d3705d-364e-4846-b3d0-e10876db24d1-kube-api-access-4h4jb\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45d3705d-364e-4846-b3d0-e10876db24d1-serving-cert\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599329    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-trusted-ca\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/109e5587-86c1-487f-8166-f8e29284cfdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.599668 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-tmp\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.600000 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.599671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-config\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.600105 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.600085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/45d3705d-364e-4846-b3d0-e10876db24d1-trusted-ca\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.600163 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.600135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/109e5587-86c1-487f-8166-f8e29284cfdf-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.601821 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.601806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45d3705d-364e-4846-b3d0-e10876db24d1-serving-cert\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.608596 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.608577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h4jb\" (UniqueName: \"kubernetes.io/projected/45d3705d-364e-4846-b3d0-e10876db24d1-kube-api-access-4h4jb\") pod \"console-operator-85b454874b-t55rq\" (UID: \"45d3705d-364e-4846-b3d0-e10876db24d1\") " pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.608912 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.608880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lf8\" (UniqueName: \"kubernetes.io/projected/109e5587-86c1-487f-8166-f8e29284cfdf-kube-api-access-25lf8\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:41.694887 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.694852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"
Sep 06 04:59:41.700146 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700146 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.700326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700326 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.700282    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 04:59:41.700326 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-stats-auth\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.700362    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:42.200341021 +0000 UTC m=+244.289418692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : secret "router-metrics-certs-default" not found
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5db9\" (UniqueName: \"kubernetes.io/projected/7062649b-61ff-45c2-8c19-3277489659bc-kube-api-access-w5db9\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzwjp\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.700569    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.700582    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d887db745-9w6xm: secret "image-registry-tls" not found
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.700633    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls podName:e2243778-8899-4f11-8805-91ea1f3f0b51 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:42.20061825 +0000 UTC m=+244.289695939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls") pod "image-registry-5d887db745-9w6xm" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51") : secret "image-registry-tls" not found
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-default-certificate\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-snapshots\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c23ef83-d706-4e78-92c7-2da117488b31-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t5xm\" (UniqueName: \"kubernetes.io/projected/1c23ef83-d706-4e78-92c7-2da117488b31-kube-api-access-5t5xm\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.700950 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.700871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-tmp\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.701578 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.701238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-tmp\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.701578 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.701271    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.701578 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:41.701380    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:42.20136142 +0000 UTC m=+244.290439113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : configmap references non-existent config key: service-ca.crt
Sep 06 04:59:41.701578 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.701551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c23ef83-d706-4e78-92c7-2da117488b31-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.701877 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.701845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1c23ef83-d706-4e78-92c7-2da117488b31-snapshots\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.702424 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.702400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.703888 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.703863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-stats-auth\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.704050 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.704027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.704291 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.704224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.704291 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.704251    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c23ef83-d706-4e78-92c7-2da117488b31-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.704409 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.704392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-default-certificate\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.709430 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.709406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t5xm\" (UniqueName: \"kubernetes.io/projected/1c23ef83-d706-4e78-92c7-2da117488b31-kube-api-access-5t5xm\") pod \"insights-operator-76bcd6ccf5-gjfnp\" (UID: \"1c23ef83-d706-4e78-92c7-2da117488b31\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.709522 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.709485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.710282 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.710265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5db9\" (UniqueName: \"kubernetes.io/projected/7062649b-61ff-45c2-8c19-3277489659bc-kube-api-access-w5db9\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:41.710865 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.710845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzwjp\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:41.797430 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.797389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:41.807694 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.807672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp"
Sep 06 04:59:41.968105 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:41.968027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5"]
Sep 06 04:59:41.971205 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:41.971181    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc580bd12_8c7c_434a_8a45_a003ca171311.slice/crio-62120e28f8a8faa2a974774b573ab11bbf53cce6bc468a5adb8971c72001a177 WatchSource:0}: Error finding container 62120e28f8a8faa2a974774b573ab11bbf53cce6bc468a5adb8971c72001a177: Status 404 returned error can't find the container with id 62120e28f8a8faa2a974774b573ab11bbf53cce6bc468a5adb8971c72001a177
Sep 06 04:59:42.073819 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.073785    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-t55rq"]
Sep 06 04:59:42.076801 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:42.076776    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d3705d_364e_4846_b3d0_e10876db24d1.slice/crio-4e9639cbe1a5aba461f9f3077de108a21d744031ae9e6b50ab3dacfa922618d5 WatchSource:0}: Error finding container 4e9639cbe1a5aba461f9f3077de108a21d744031ae9e6b50ab3dacfa922618d5: Status 404 returned error can't find the container with id 4e9639cbe1a5aba461f9f3077de108a21d744031ae9e6b50ab3dacfa922618d5
Sep 06 04:59:42.082727 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.082690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjfnp"]
Sep 06 04:59:42.085612 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:42.085591    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c23ef83_d706_4e78_92c7_2da117488b31.slice/crio-99ea351404a4374fb2e7899a00b6e0946645469ee9293eda04a8e3f254813bdf WatchSource:0}: Error finding container 99ea351404a4374fb2e7899a00b6e0946645469ee9293eda04a8e3f254813bdf: Status 404 returned error can't find the container with id 99ea351404a4374fb2e7899a00b6e0946645469ee9293eda04a8e3f254813bdf
Sep 06 04:59:42.104998 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.104969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:42.105139 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.105079    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:42.105139 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.105132    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls podName:109e5587-86c1-487f-8166-f8e29284cfdf nodeName:}" failed. No retries permitted until 2026-09-06 04:59:43.105116467 +0000 UTC m=+245.194194141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-kpp95" (UID: "109e5587-86c1-487f-8166-f8e29284cfdf") : secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:42.205597 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.205563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:42.205743 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.205608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:42.205743 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.205661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205750    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205792    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:43.205774563 +0000 UTC m=+245.294852250 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : configmap references non-existent config key: service-ca.crt
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205797    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205813    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d887db745-9w6xm: secret "image-registry-tls" not found
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205820    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:43.205801128 +0000 UTC m=+245.294878815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : secret "router-metrics-certs-default" not found
Sep 06 04:59:42.205865 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:42.205854    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls podName:e2243778-8899-4f11-8805-91ea1f3f0b51 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:43.205843287 +0000 UTC m=+245.294920995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls") pod "image-registry-5d887db745-9w6xm" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51") : secret "image-registry-tls" not found
Sep 06 04:59:42.212958 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.212928    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5" event={"ID":"c580bd12-8c7c-434a-8a45-a003ca171311","Type":"ContainerStarted","Data":"62120e28f8a8faa2a974774b573ab11bbf53cce6bc468a5adb8971c72001a177"}
Sep 06 04:59:42.214072 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.214052    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp" event={"ID":"1c23ef83-d706-4e78-92c7-2da117488b31","Type":"ContainerStarted","Data":"99ea351404a4374fb2e7899a00b6e0946645469ee9293eda04a8e3f254813bdf"}
Sep 06 04:59:42.214999 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:42.214980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t55rq" event={"ID":"45d3705d-364e-4846-b3d0-e10876db24d1","Type":"ContainerStarted","Data":"4e9639cbe1a5aba461f9f3077de108a21d744031ae9e6b50ab3dacfa922618d5"}
Sep 06 04:59:43.115082 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:43.114449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:43.115082 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.114652    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:43.115082 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.114722    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls podName:109e5587-86c1-487f-8166-f8e29284cfdf nodeName:}" failed. No retries permitted until 2026-09-06 04:59:45.114703083 +0000 UTC m=+247.203780760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-kpp95" (UID: "109e5587-86c1-487f-8166-f8e29284cfdf") : secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:43.215418 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:43.215110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:43.215418 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:43.215179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:43.215418 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215373    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 04:59:43.215418 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215397    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d887db745-9w6xm: secret "image-registry-tls" not found
Sep 06 04:59:43.215418 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215415    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 04:59:43.215810 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215467    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls podName:e2243778-8899-4f11-8805-91ea1f3f0b51 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:45.215445071 +0000 UTC m=+247.304522783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls") pod "image-registry-5d887db745-9w6xm" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51") : secret "image-registry-tls" not found
Sep 06 04:59:43.215810 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:43.215492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:43.215810 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:45.215546138 +0000 UTC m=+247.304623815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : secret "router-metrics-certs-default" not found
Sep 06 04:59:43.215810 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:43.215628    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:45.215616859 +0000 UTC m=+247.304694530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : configmap references non-existent config key: service-ca.crt
Sep 06 04:59:44.223878 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:44.223831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5" event={"ID":"c580bd12-8c7c-434a-8a45-a003ca171311","Type":"ContainerStarted","Data":"fbb5a559c6ce215f0a4375e5cfe22f7e81722f1686c8bd5e841d75be6b766322"}
Sep 06 04:59:44.239281 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:44.239223    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-gntt5" podStartSLOduration=1.8279663130000001 podStartE2EDuration="3.239202815s" totalImagesPullingTime="1.411236502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:59:44.238105692 +0000 UTC m=+246.327183386" watchObservedRunningTime="2026-09-06 04:59:44.239202815 +0000 UTC m=+246.328280508"
Sep 06 04:59:45.130862 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.130821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:45.131062 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.130998    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:45.131125 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.131069    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls podName:109e5587-86c1-487f-8166-f8e29284cfdf nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.131051194 +0000 UTC m=+251.220128866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-kpp95" (UID: "109e5587-86c1-487f-8166-f8e29284cfdf") : secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:45.229229 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.229197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp" event={"ID":"1c23ef83-d706-4e78-92c7-2da117488b31","Type":"ContainerStarted","Data":"4f931e377b51e5c08ea1c32f0c9c67e46c14fe059f3b3f3e2acab6cee1082d52"}
Sep 06 04:59:45.231027 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231005    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/0.log"
Sep 06 04:59:45.231147 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231046    2576 generic.go:356] "Generic (PLEG): container finished" podID="45d3705d-364e-4846-b3d0-e10876db24d1" containerID="71d65e81fb0502d38e1c44135b5430691e31d71b4ca5965a0816c36d8f163736" exitCode=255
Sep 06 04:59:45.231147 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t55rq" event={"ID":"45d3705d-364e-4846-b3d0-e10876db24d1","Type":"ContainerDied","Data":"71d65e81fb0502d38e1c44135b5430691e31d71b4ca5965a0816c36d8f163736"}
Sep 06 04:59:45.231330 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:45.231460 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231351    2576 scope.go:117] "RemoveContainer" containerID="71d65e81fb0502d38e1c44135b5430691e31d71b4ca5965a0816c36d8f163736"
Sep 06 04:59:45.231460 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:45.231460 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231412    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 04:59:45.231460 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231428    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d887db745-9w6xm: secret "image-registry-tls" not found
Sep 06 04:59:45.231460 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.231450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:45.231697 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231480    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls podName:e2243778-8899-4f11-8805-91ea1f3f0b51 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.23146021 +0000 UTC m=+251.320537881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls") pod "image-registry-5d887db745-9w6xm" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51") : secret "image-registry-tls" not found
Sep 06 04:59:45.231697 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231538    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 04:59:45.231697 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231540    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.231524102 +0000 UTC m=+251.320601786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : configmap references non-existent config key: service-ca.crt
Sep 06 04:59:45.231697 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.231593    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.231578418 +0000 UTC m=+251.320656092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : secret "router-metrics-certs-default" not found
Sep 06 04:59:45.245113 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.245057    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp" podStartSLOduration=1.846716731 podStartE2EDuration="4.245036931s" totalImagesPullingTime="2.3983202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:59:45.244187564 +0000 UTC m=+247.333265260" watchObservedRunningTime="2026-09-06 04:59:45.245036931 +0000 UTC m=+247.334114627"
Sep 06 04:59:45.765268 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.765236    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jzvzj"]
Sep 06 04:59:45.768428 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.768408    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.770751 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.770725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 06 04:59:45.771047 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.771030    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qr8nl\""
Sep 06 04:59:45.771159 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.771078    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 06 04:59:45.782023 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.781996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzvzj"]
Sep 06 04:59:45.836476 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.836442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxvv8\" (UniqueName: \"kubernetes.io/projected/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-api-access-cxvv8\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.836630 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.836504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b7b8e6-29c0-4b88-aea8-4934920a6305-crio-socket\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.836630 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.836533    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.836716 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.836672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b7b8e6-29c0-4b88-aea8-4934920a6305-data-volume\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.836750 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.836713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.937891 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.937851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxvv8\" (UniqueName: \"kubernetes.io/projected/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-api-access-cxvv8\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.937891 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.937895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b7b8e6-29c0-4b88-aea8-4934920a6305-crio-socket\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.937936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.937993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b7b8e6-29c0-4b88-aea8-4934920a6305-data-volume\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.938016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.938055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29b7b8e6-29c0-4b88-aea8-4934920a6305-crio-socket\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.938083    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.938127    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:45.938152 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.938156    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:46.438136097 +0000 UTC m=+248.527213785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:45.938450 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:45.938178    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:46.438163742 +0000 UTC m=+248.527241412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:45.938450 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.938323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29b7b8e6-29c0-4b88-aea8-4934920a6305-data-volume\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:45.945958 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:45.945929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxvv8\" (UniqueName: \"kubernetes.io/projected/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-api-access-cxvv8\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:46.236174 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.236146    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/1.log"
Sep 06 04:59:46.236652 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.236571    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/0.log"
Sep 06 04:59:46.236652 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.236612    2576 generic.go:356] "Generic (PLEG): container finished" podID="45d3705d-364e-4846-b3d0-e10876db24d1" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda" exitCode=255
Sep 06 04:59:46.236752 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.236690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t55rq" event={"ID":"45d3705d-364e-4846-b3d0-e10876db24d1","Type":"ContainerDied","Data":"5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"}
Sep 06 04:59:46.237676 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.237657    2576 scope.go:117] "RemoveContainer" containerID="71d65e81fb0502d38e1c44135b5430691e31d71b4ca5965a0816c36d8f163736"
Sep 06 04:59:46.237773 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.237756    2576 scope.go:117] "RemoveContainer" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"
Sep 06 04:59:46.237982 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:46.237957    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 04:59:46.441505 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.441470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:46.441726 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:46.441603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:46.441726 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:46.441613    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.441726 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:46.441689    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:47.441669707 +0000 UTC m=+249.530747392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:46.441943 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:46.441747    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:46.441943 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:46.441799    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:47.441782787 +0000 UTC m=+249.530860463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:47.240964 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:47.240935    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/1.log"
Sep 06 04:59:47.241382 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:47.241261    2576 scope.go:117] "RemoveContainer" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"
Sep 06 04:59:47.241428 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:47.241416    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 04:59:47.450480 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:47.450441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:47.450659 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:47.450512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:47.450659 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:47.450583    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:47.450659 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:47.450597    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:47.450659 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:47.450657    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.450637821 +0000 UTC m=+251.539715492 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:47.450809 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:47.450682    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:49.450675036 +0000 UTC m=+251.539752706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:47.735107 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:47.735076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jwsh9_544f66cd-a01a-480b-aea0-23c82fc314c0/dns-node-resolver/0.log"
Sep 06 04:59:48.931785 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:48.931758    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rgc5h_a452e6b1-85c2-485d-8d50-3434e01caae4/node-ca/0.log"
Sep 06 04:59:49.165259 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.165219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:49.165462 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.165354    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:49.165462 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.165422    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls podName:109e5587-86c1-487f-8166-f8e29284cfdf nodeName:}" failed. No retries permitted until 2026-09-06 04:59:57.165403396 +0000 UTC m=+259.254481081 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-kpp95" (UID: "109e5587-86c1-487f-8166-f8e29284cfdf") : secret "cluster-monitoring-operator-tls" not found
Sep 06 04:59:49.266473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.266378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:49.266473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.266446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:49.266473 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.266476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266564    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266574    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266576    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d887db745-9w6xm: secret "image-registry-tls" not found
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266563    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:57.266541944 +0000 UTC m=+259.355619615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : configmap references non-existent config key: service-ca.crt
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266716    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs podName:7062649b-61ff-45c2-8c19-3277489659bc nodeName:}" failed. No retries permitted until 2026-09-06 04:59:57.266705651 +0000 UTC m=+259.355783322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs") pod "router-default-57b77cb778-p6mrm" (UID: "7062649b-61ff-45c2-8c19-3277489659bc") : secret "router-metrics-certs-default" not found
Sep 06 04:59:49.266754 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.266727    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls podName:e2243778-8899-4f11-8805-91ea1f3f0b51 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:57.266721293 +0000 UTC m=+259.355798964 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls") pod "image-registry-5d887db745-9w6xm" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51") : secret "image-registry-tls" not found
Sep 06 04:59:49.468719 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.468681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:49.468941 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.468752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:49.468941 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.468826    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:49.468941 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.468829    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:49.468941 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.468926    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:53.468878798 +0000 UTC m=+255.557956486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:49.469098 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:49.468965    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 04:59:53.468951986 +0000 UTC m=+255.558029687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:49.531520 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:49.531444    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hzl5_80a2d8d2-7870-4e16-aaee-c242d7ea3c0c/serve-healthcheck-canary/0.log"
Sep 06 04:59:50.276484 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:50.276450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 04:59:50.276821 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:50.276554    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 06 04:59:50.276821 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:50.276615    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs podName:4f3f6321-5aaf-4656-ad4c-04109aeba114 nodeName:}" failed. No retries permitted until 2026-09-06 05:01:52.276601067 +0000 UTC m=+374.365678741 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs") pod "network-metrics-daemon-r7t92" (UID: "4f3f6321-5aaf-4656-ad4c-04109aeba114") : secret "metrics-daemon-secret" not found
Sep 06 04:59:51.797794 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:51.797763    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:51.798197 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:51.797813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 04:59:51.798197 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:51.798148    2576 scope.go:117] "RemoveContainer" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"
Sep 06 04:59:51.798313 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:51.798295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 04:59:53.505137 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:53.505096    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:53.505607 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:53.505184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 04:59:53.505607 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:53.505232    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 06 04:59:53.505607 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:53.505306    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:53.505607 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:53.505315    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 05:00:01.505292465 +0000 UTC m=+263.594370139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : configmap "kube-rbac-proxy" not found
Sep 06 04:59:53.505607 ip-10-0-130-133 kubenswrapper[2576]: E0906 04:59:53.505353    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls podName:29b7b8e6-29c0-4b88-aea8-4934920a6305 nodeName:}" failed. No retries permitted until 2026-09-06 05:00:01.505342084 +0000 UTC m=+263.594419822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jzvzj" (UID: "29b7b8e6-29c0-4b88-aea8-4934920a6305") : secret "insights-runtime-extractor-tls" not found
Sep 06 04:59:57.236633 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.236591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:57.239213 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.239182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/109e5587-86c1-487f-8166-f8e29284cfdf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-kpp95\" (UID: \"109e5587-86c1-487f-8166-f8e29284cfdf\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:57.300955 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.300889    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"
Sep 06 04:59:57.337590 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.337544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:57.337783 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.337602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:57.337783 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.337630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:57.338306 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.338280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7062649b-61ff-45c2-8c19-3277489659bc-service-ca-bundle\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:57.340164 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.340141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"image-registry-5d887db745-9w6xm\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") " pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:57.340278 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.340247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7062649b-61ff-45c2-8c19-3277489659bc-metrics-certs\") pod \"router-default-57b77cb778-p6mrm\" (UID: \"7062649b-61ff-45c2-8c19-3277489659bc\") " pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:57.413439 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.413395    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:57.423205 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.423183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:57.572618 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.572594    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95"]
Sep 06 04:59:57.575029 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:57.575004    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109e5587_86c1_487f_8166_f8e29284cfdf.slice/crio-137ded2f8c6b1acf251f60f024f7722861c4a8f4d6ca6598beaaee98cf0de5bc WatchSource:0}: Error finding container 137ded2f8c6b1acf251f60f024f7722861c4a8f4d6ca6598beaaee98cf0de5bc: Status 404 returned error can't find the container with id 137ded2f8c6b1acf251f60f024f7722861c4a8f4d6ca6598beaaee98cf0de5bc
Sep 06 04:59:57.689211 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.689190    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57b77cb778-p6mrm"]
Sep 06 04:59:57.691513 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:57.691487    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7062649b_61ff_45c2_8c19_3277489659bc.slice/crio-367ac88ee57948ca2f3b06a25b9ec6484994d55467755204e2ee7d11d2125536 WatchSource:0}: Error finding container 367ac88ee57948ca2f3b06a25b9ec6484994d55467755204e2ee7d11d2125536: Status 404 returned error can't find the container with id 367ac88ee57948ca2f3b06a25b9ec6484994d55467755204e2ee7d11d2125536
Sep 06 04:59:57.713006 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:57.712980    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 04:59:57.723866 ip-10-0-130-133 kubenswrapper[2576]: W0906 04:59:57.723844    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2243778_8899_4f11_8805_91ea1f3f0b51.slice/crio-56707fa53bfc5ab44c4ef7ffcec47d86797df89da95b06a398b24f96abc45e1b WatchSource:0}: Error finding container 56707fa53bfc5ab44c4ef7ffcec47d86797df89da95b06a398b24f96abc45e1b: Status 404 returned error can't find the container with id 56707fa53bfc5ab44c4ef7ffcec47d86797df89da95b06a398b24f96abc45e1b
Sep 06 04:59:58.276287 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.276246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" event={"ID":"e2243778-8899-4f11-8805-91ea1f3f0b51","Type":"ContainerStarted","Data":"d0bff9af0c2713ee3728761a3b394c6f592048f89f9286026ecd0b733d1bbcb9"}
Sep 06 04:59:58.276718 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.276310    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 04:59:58.276718 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.276328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" event={"ID":"e2243778-8899-4f11-8805-91ea1f3f0b51","Type":"ContainerStarted","Data":"56707fa53bfc5ab44c4ef7ffcec47d86797df89da95b06a398b24f96abc45e1b"}
Sep 06 04:59:58.277843 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.277820    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57b77cb778-p6mrm" event={"ID":"7062649b-61ff-45c2-8c19-3277489659bc","Type":"ContainerStarted","Data":"51f20a25e0c9477afded6296109eb696ce9887c8f9104056026c1b9d1888a1f1"}
Sep 06 04:59:58.277927 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.277857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57b77cb778-p6mrm" event={"ID":"7062649b-61ff-45c2-8c19-3277489659bc","Type":"ContainerStarted","Data":"367ac88ee57948ca2f3b06a25b9ec6484994d55467755204e2ee7d11d2125536"}
Sep 06 04:59:58.278920 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.278890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95" event={"ID":"109e5587-86c1-487f-8166-f8e29284cfdf","Type":"ContainerStarted","Data":"137ded2f8c6b1acf251f60f024f7722861c4a8f4d6ca6598beaaee98cf0de5bc"}
Sep 06 04:59:58.294737 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.294683    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" podStartSLOduration=17.294665934 podStartE2EDuration="17.294665934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:59:58.293387314 +0000 UTC m=+260.382465009" watchObservedRunningTime="2026-09-06 04:59:58.294665934 +0000 UTC m=+260.383743631"
Sep 06 04:59:58.310369 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.310317    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57b77cb778-p6mrm" podStartSLOduration=17.310301933 podStartE2EDuration="17.310301933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:59:58.309118714 +0000 UTC m=+260.398196409" watchObservedRunningTime="2026-09-06 04:59:58.310301933 +0000 UTC m=+260.399379628"
Sep 06 04:59:58.413773 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.413742    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:58.416990 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:58.416968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:59.285121 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:59.285087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95" event={"ID":"109e5587-86c1-487f-8166-f8e29284cfdf","Type":"ContainerStarted","Data":"f3feaafa9de677b859bc77c3cb0e2537a9c29656b9a554be4ee1e036c257e799"}
Sep 06 04:59:59.285622 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:59.285601    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:59.286781 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:59.286765    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57b77cb778-p6mrm"
Sep 06 04:59:59.299527 ip-10-0-130-133 kubenswrapper[2576]: I0906 04:59:59.299454    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-kpp95" podStartSLOduration=16.696623146 podStartE2EDuration="18.29943228s" totalImagesPullingTime="1.602809134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 04:59:59.299372386 +0000 UTC m=+261.388450080" watchObservedRunningTime="2026-09-06 04:59:59.29943228 +0000 UTC m=+261.388509973"
Sep 06 05:00:01.573209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.573164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 05:00:01.573616 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.573238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 05:00:01.573748 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.573730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29b7b8e6-29c0-4b88-aea8-4934920a6305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 05:00:01.575574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.575553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29b7b8e6-29c0-4b88-aea8-4934920a6305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jzvzj\" (UID: \"29b7b8e6-29c0-4b88-aea8-4934920a6305\") " pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 05:00:01.677074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.677021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jzvzj"
Sep 06 05:00:01.946189 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:01.946149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jzvzj"]
Sep 06 05:00:01.950289 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:00:01.950251    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b7b8e6_29c0_4b88_aea8_4934920a6305.slice/crio-20c86fc95c853bc0f3bd28b886f9453a4c55f512096632d63716753f06929b3a WatchSource:0}: Error finding container 20c86fc95c853bc0f3bd28b886f9453a4c55f512096632d63716753f06929b3a: Status 404 returned error can't find the container with id 20c86fc95c853bc0f3bd28b886f9453a4c55f512096632d63716753f06929b3a
Sep 06 05:00:02.296205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:02.296123    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzvzj" event={"ID":"29b7b8e6-29c0-4b88-aea8-4934920a6305","Type":"ContainerStarted","Data":"bc3c28e4ad2120d377a2c00dbbdf094bef46b286b642d26296e227b0fb8070ee"}
Sep 06 05:00:02.296205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:02.296177    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzvzj" event={"ID":"29b7b8e6-29c0-4b88-aea8-4934920a6305","Type":"ContainerStarted","Data":"20c86fc95c853bc0f3bd28b886f9453a4c55f512096632d63716753f06929b3a"}
Sep 06 05:00:04.304501 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:04.304465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzvzj" event={"ID":"29b7b8e6-29c0-4b88-aea8-4934920a6305","Type":"ContainerStarted","Data":"7278f543bbdd8ea3eccccd96c95c43f98a0232ab3af6bf8bf4433785f411ae0e"}
Sep 06 05:00:05.310120 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:05.310033    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jzvzj" event={"ID":"29b7b8e6-29c0-4b88-aea8-4934920a6305","Type":"ContainerStarted","Data":"1bb13b3c3bd25061b006239bbb90e670a8cec6c8f9bb7f3cc25bff2a759f9664"}
Sep 06 05:00:05.328340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:05.328287    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jzvzj" podStartSLOduration=17.417052009 podStartE2EDuration="20.328269714s" totalImagesPullingTime="2.911217705s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 04:59:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:05.326523056 +0000 UTC m=+267.415600762" watchObservedRunningTime="2026-09-06 05:00:05.328269714 +0000 UTC m=+267.417347406"
Sep 06 05:00:05.380354 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:05.380327    2576 scope.go:117] "RemoveContainer" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"
Sep 06 05:00:06.314738 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.314710    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:00:06.315171 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.315150    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/1.log"
Sep 06 05:00:06.315208 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.315179    2576 generic.go:356] "Generic (PLEG): container finished" podID="45d3705d-364e-4846-b3d0-e10876db24d1" containerID="a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd" exitCode=255
Sep 06 05:00:06.315279 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.315256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t55rq" event={"ID":"45d3705d-364e-4846-b3d0-e10876db24d1","Type":"ContainerDied","Data":"a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd"}
Sep 06 05:00:06.315354 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.315343    2576 scope.go:117] "RemoveContainer" containerID="5e3f1367897bb9a0702b67ced9f7af9b1eb27010e3ae68a1b27889ca8564ffda"
Sep 06 05:00:06.315694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:06.315673    2576 scope.go:117] "RemoveContainer" containerID="a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd"
Sep 06 05:00:06.315845 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:00:06.315827    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 05:00:07.183720 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.183635    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 05:00:07.219881 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.219849    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7689454c6c-qvsv8"]
Sep 06 05:00:07.224045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.224028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.236444 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.236418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7689454c6c-qvsv8"]
Sep 06 05:00:07.318525 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-certificates\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-image-registry-private-configuration\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-installation-pull-secrets\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-trusted-ca\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-bound-sa-token\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318729    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/592d3736-2b3a-4ee9-86ca-6c100cf66da2-ca-trust-extracted\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-tls\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.319009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.318831    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgkt\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-kube-api-access-vrgkt\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.320005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.319990    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:00:07.419316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-trusted-ca\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-bound-sa-token\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/592d3736-2b3a-4ee9-86ca-6c100cf66da2-ca-trust-extracted\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-tls\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgkt\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-kube-api-access-vrgkt\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-certificates\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419755 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-image-registry-private-configuration\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.419755 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.419606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-installation-pull-secrets\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.420050 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.420018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/592d3736-2b3a-4ee9-86ca-6c100cf66da2-ca-trust-extracted\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.420316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.420293    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-trusted-ca\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.420565 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.420545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-certificates\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.422155 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.422135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-image-registry-private-configuration\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.422234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.422134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/592d3736-2b3a-4ee9-86ca-6c100cf66da2-installation-pull-secrets\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.422234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.422183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-registry-tls\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.430297 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.430277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgkt\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-kube-api-access-vrgkt\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.430372 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.430341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/592d3736-2b3a-4ee9-86ca-6c100cf66da2-bound-sa-token\") pod \"image-registry-7689454c6c-qvsv8\" (UID: \"592d3736-2b3a-4ee9-86ca-6c100cf66da2\") " pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.532409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.532315    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:07.801498 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:07.801403    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7689454c6c-qvsv8"]
Sep 06 05:00:07.803465 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:00:07.803435    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592d3736_2b3a_4ee9_86ca_6c100cf66da2.slice/crio-477176dcb751b87e094ee5985df3526dd027dff928baaf81901d318903f72143 WatchSource:0}: Error finding container 477176dcb751b87e094ee5985df3526dd027dff928baaf81901d318903f72143: Status 404 returned error can't find the container with id 477176dcb751b87e094ee5985df3526dd027dff928baaf81901d318903f72143
Sep 06 05:00:08.324620 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:08.324578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8" event={"ID":"592d3736-2b3a-4ee9-86ca-6c100cf66da2","Type":"ContainerStarted","Data":"492dcf46a49fb71ddcb17321d43109d36dcbe2a9aaffbc252481dd1149a9f610"}
Sep 06 05:00:08.325035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:08.324632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8" event={"ID":"592d3736-2b3a-4ee9-86ca-6c100cf66da2","Type":"ContainerStarted","Data":"477176dcb751b87e094ee5985df3526dd027dff928baaf81901d318903f72143"}
Sep 06 05:00:08.325035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:08.324672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:08.344225 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:08.344182    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8" podStartSLOduration=1.34416606 podStartE2EDuration="1.34416606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:08.342189229 +0000 UTC m=+270.431266922" watchObservedRunningTime="2026-09-06 05:00:08.34416606 +0000 UTC m=+270.433243770"
Sep 06 05:00:11.798552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:11.798499    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 05:00:11.799000 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:11.798573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 05:00:11.799000 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:11.798886    2576 scope.go:117] "RemoveContainer" containerID="a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd"
Sep 06 05:00:11.799096 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:00:11.799075    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 05:00:14.057045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.057009    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q2zb8"]
Sep 06 05:00:14.061919 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.061885    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.064316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.064295    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 06 05:00:14.064751 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.064732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 06 05:00:14.064852 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.064783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 06 05:00:14.064852 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.064817    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 06 05:00:14.065701 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.065684    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6z6cl\""
Sep 06 05:00:14.171035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.170998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-textfile\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-tls\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-metrics-client-ca\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-wtmp\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-root\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171218    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-sys\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhmgf\" (UniqueName: \"kubernetes.io/projected/5ebdbc2a-07e3-4051-8081-65588f2803d3-kube-api-access-nhmgf\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.171374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.171308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.272793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-textfile\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.272856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-tls\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.272966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-metrics-client-ca\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.272995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-wtmp\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-root\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-sys\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhmgf\" (UniqueName: \"kubernetes.io/projected/5ebdbc2a-07e3-4051-8081-65588f2803d3-kube-api-access-nhmgf\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.273883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-accelerators-collector-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.274157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-textfile\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.274855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.274589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-root\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.275504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.275100    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ebdbc2a-07e3-4051-8081-65588f2803d3-metrics-client-ca\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.275504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.275248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-wtmp\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.278393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.275887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ebdbc2a-07e3-4051-8081-65588f2803d3-sys\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.278393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.278147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.279627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.279600    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5ebdbc2a-07e3-4051-8081-65588f2803d3-node-exporter-tls\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.283339 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.283315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhmgf\" (UniqueName: \"kubernetes.io/projected/5ebdbc2a-07e3-4051-8081-65588f2803d3-kube-api-access-nhmgf\") pod \"node-exporter-q2zb8\" (UID: \"5ebdbc2a-07e3-4051-8081-65588f2803d3\") " pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.370861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:14.370772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q2zb8"
Sep 06 05:00:14.382281 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:00:14.382257    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ebdbc2a_07e3_4051_8081_65588f2803d3.slice/crio-8f0f1837939c13a0f2c4d9c57faa4f28666bb8f0838d0c1f6b3024988b45ac25 WatchSource:0}: Error finding container 8f0f1837939c13a0f2c4d9c57faa4f28666bb8f0838d0c1f6b3024988b45ac25: Status 404 returned error can't find the container with id 8f0f1837939c13a0f2c4d9c57faa4f28666bb8f0838d0c1f6b3024988b45ac25
Sep 06 05:00:15.349920 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:15.349868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2zb8" event={"ID":"5ebdbc2a-07e3-4051-8081-65588f2803d3","Type":"ContainerStarted","Data":"8f0f1837939c13a0f2c4d9c57faa4f28666bb8f0838d0c1f6b3024988b45ac25"}
Sep 06 05:00:16.354610 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:16.354572    2576 generic.go:356] "Generic (PLEG): container finished" podID="5ebdbc2a-07e3-4051-8081-65588f2803d3" containerID="e6dd892077b33b33b9cd4e805febb69a8f3ad86fd3a2247eb67f69656d5e0a73" exitCode=0
Sep 06 05:00:16.355022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:16.354632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2zb8" event={"ID":"5ebdbc2a-07e3-4051-8081-65588f2803d3","Type":"ContainerDied","Data":"e6dd892077b33b33b9cd4e805febb69a8f3ad86fd3a2247eb67f69656d5e0a73"}
Sep 06 05:00:17.190088 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:17.190051    2576 patch_prober.go:28] interesting pod/image-registry-5d887db745-9w6xm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 06 05:00:17.190250 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:17.190108    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:00:17.360167 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:17.360137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2zb8" event={"ID":"5ebdbc2a-07e3-4051-8081-65588f2803d3","Type":"ContainerStarted","Data":"2fe4e7ffab292507ebc545de389d203f5e74bd914c60c9bb58c670deaed9170a"}
Sep 06 05:00:17.360555 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:17.360189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q2zb8" event={"ID":"5ebdbc2a-07e3-4051-8081-65588f2803d3","Type":"ContainerStarted","Data":"01b944b3eb6ea1cc0ea6720f2bfc01ba539c9f50a0ea1cfe0b3fdc37e3f56c8b"}
Sep 06 05:00:17.378019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:17.377966    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q2zb8" podStartSLOduration=2.268060648 podStartE2EDuration="3.377948784s" totalImagesPullingTime="1.040527081s" totalInitContainerRuntime="69.361055ms" isStatefulPod=true podCreationTimestamp="2026-09-06 05:00:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:17.37663126 +0000 UTC m=+279.465708975" watchObservedRunningTime="2026-09-06 05:00:17.377948784 +0000 UTC m=+279.467026476"
Sep 06 05:00:18.992067 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:00:18.992016    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-v47hr" podUID="12b770a7-9c8d-4f77-8d3a-0b8f98ad91db"
Sep 06 05:00:19.366374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:19.366279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:20.249973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.249938    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:00:20.253713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.253694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.256176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.256128    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h64vm\""
Sep 06 05:00:20.256284 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.256219    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 06 05:00:20.256474 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.256456    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 06 05:00:20.256572 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.256460    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3njndatvdmjim\""
Sep 06 05:00:20.256572 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.256528    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 06 05:00:20.257400 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257382    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 06 05:00:20.257503 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257444    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 06 05:00:20.257503 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257485    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 06 05:00:20.257614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257495    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 06 05:00:20.257614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257515    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 06 05:00:20.257614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257610    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 06 05:00:20.257843 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 06 05:00:20.257942 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.257844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 06 05:00:20.258104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.258078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 06 05:00:20.258504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.258488    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 06 05:00:20.260384 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.260363    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 06 05:00:20.262378 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.262358    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 06 05:00:20.270892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.270871    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:00:20.425475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425432    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jb9q\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425657    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425742    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.425921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.426166 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.426166 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.426166 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.426166 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.426166 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.425995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526587 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526587 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526587 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526826 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526826 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526960 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526846    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526960 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.526960 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.526992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.527022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527268 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.527191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527327 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.527311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.527376 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.527353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528174 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jb9q\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528344 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528470 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528544 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528452    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528544 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528646 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528646 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.528892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.528202    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.530484 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.530459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.530586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.530499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.531020 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.530995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.531680 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.531653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.532306 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.532265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.532733 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.532695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.532933 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.532874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.533504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.533344    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.533504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.533350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.533504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.533458    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.533688 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.533598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.534080 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.534051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.534705 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.534679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.534812 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.534795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.536176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.536158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jb9q\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q\") pod \"prometheus-k8s-0\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.564213 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.564180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:20.840085 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:20.840054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:00:20.842741 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:00:20.842718    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda78581b1_fe0a_4c52_9e03_1fccb9c3acbf.slice/crio-7c3d2852e98165f3bfee7b1bb84c483baa02b5c0bfaf16f47a12b64bd38e8b64 WatchSource:0}: Error finding container 7c3d2852e98165f3bfee7b1bb84c483baa02b5c0bfaf16f47a12b64bd38e8b64: Status 404 returned error can't find the container with id 7c3d2852e98165f3bfee7b1bb84c483baa02b5c0bfaf16f47a12b64bd38e8b64
Sep 06 05:00:21.374359 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:21.374298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"7c3d2852e98165f3bfee7b1bb84c483baa02b5c0bfaf16f47a12b64bd38e8b64"}
Sep 06 05:00:22.378755 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.378721    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="6310d05cd348b5b1d249e993ac456ddc991878e2afd6e4399f18cb486854b58f" exitCode=0
Sep 06 05:00:22.379181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.378775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"6310d05cd348b5b1d249e993ac456ddc991878e2afd6e4399f18cb486854b58f"}
Sep 06 05:00:22.652810 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.652715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:22.655205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.655184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/12b770a7-9c8d-4f77-8d3a-0b8f98ad91db-metrics-tls\") pod \"dns-default-v47hr\" (UID: \"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db\") " pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:22.669715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.669687    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-prh2m\""
Sep 06 05:00:22.677622 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.677580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:22.948304 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:22.948269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v47hr"]
Sep 06 05:00:22.951545 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:00:22.951510    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b770a7_9c8d_4f77_8d3a_0b8f98ad91db.slice/crio-31eeae7789af7fcd86d8b5ec5d1187a3b067f5d97562374e775613142a5e59f1 WatchSource:0}: Error finding container 31eeae7789af7fcd86d8b5ec5d1187a3b067f5d97562374e775613142a5e59f1: Status 404 returned error can't find the container with id 31eeae7789af7fcd86d8b5ec5d1187a3b067f5d97562374e775613142a5e59f1
Sep 06 05:00:23.384322 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:23.384236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v47hr" event={"ID":"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db","Type":"ContainerStarted","Data":"31eeae7789af7fcd86d8b5ec5d1187a3b067f5d97562374e775613142a5e59f1"}
Sep 06 05:00:24.380726 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:24.380692    2576 scope.go:117] "RemoveContainer" containerID="a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd"
Sep 06 05:00:24.381683 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:00:24.381651    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-t55rq_openshift-console-operator(45d3705d-364e-4846-b3d0-e10876db24d1)\"" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1"
Sep 06 05:00:26.397823 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.397787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"5c27f25a602dbc53073092b6a8d57954e95afe775d874571e5435da5f4d31f1a"}
Sep 06 05:00:26.398293 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.397875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"f08b36a911cc6823eee394fc94c52d5bf32db210aab046284761fa8434c09d37"}
Sep 06 05:00:26.399790 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.399761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v47hr" event={"ID":"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db","Type":"ContainerStarted","Data":"57ad2b81fc74061daad6d78b58db4dc167904e1ce70d1591fb270fb985a551d9"}
Sep 06 05:00:26.399927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.399815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v47hr" event={"ID":"12b770a7-9c8d-4f77-8d3a-0b8f98ad91db","Type":"ContainerStarted","Data":"8d08a9962e3d068db463e3d3cfb094f0e7afb56b81c957aa8facb69378c9130e"}
Sep 06 05:00:26.400058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.400040    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:26.416288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:26.416240    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-v47hr" podStartSLOduration=251.730261889 podStartE2EDuration="4m14.416224702s" totalImagesPullingTime="2.685962813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:26.415035377 +0000 UTC m=+288.504113072" watchObservedRunningTime="2026-09-06 05:00:26.416224702 +0000 UTC m=+288.505302395"
Sep 06 05:00:27.188978 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:27.188942    2576 patch_prober.go:28] interesting pod/image-registry-5d887db745-9w6xm container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 06 05:00:27.189165 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:27.189001    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:00:27.537508 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:27.537473    2576 patch_prober.go:28] interesting pod/image-registry-7689454c6c-qvsv8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 06 05:00:27.537841 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:27.537540    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8" podUID="592d3736-2b3a-4ee9-86ca-6c100cf66da2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:00:28.410584 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:28.410554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"ba6b5c8cf6340553d9570fb71d89aa12d91644a293f1867fb03387fb899f9727"}
Sep 06 05:00:28.410764 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:28.410615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"f30d868007262c5cf8a062ff2b652ba61831952acbbac7ab0afa2c071f4cbea4"}
Sep 06 05:00:28.410764 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:28.410642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"87af86a275f6ab710587d8c3616fed2f87751ce09a64dcfd52caeaa408275761"}
Sep 06 05:00:28.410764 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:28.410666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerStarted","Data":"0964964d55eca420ffdbdf15edc1764509bc44db44cac07ac2802c33d262ae0c"}
Sep 06 05:00:28.436833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:28.436779    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.675594174 podStartE2EDuration="8.436763877s" totalImagesPullingTime="5.742591698s" totalInitContainerRuntime="18.578005ms" isStatefulPod=false podCreationTimestamp="2026-09-06 05:00:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:28.434595801 +0000 UTC m=+290.523673494" watchObservedRunningTime="2026-09-06 05:00:28.436763877 +0000 UTC m=+290.525841570"
Sep 06 05:00:29.332252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:29.332223    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7689454c6c-qvsv8"
Sep 06 05:00:30.565347 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:30.565300    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:00:31.381349 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:00:31.381297    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-r7t92" podUID="4f3f6321-5aaf-4656-ad4c-04109aeba114"
Sep 06 05:00:32.203688 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.203622    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry" containerID="cri-o://d0bff9af0c2713ee3728761a3b394c6f592048f89f9286026ecd0b733d1bbcb9" gracePeriod=30
Sep 06 05:00:32.427222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.427189    2576 generic.go:356] "Generic (PLEG): container finished" podID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerID="d0bff9af0c2713ee3728761a3b394c6f592048f89f9286026ecd0b733d1bbcb9" exitCode=0
Sep 06 05:00:32.427380 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.427258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" event={"ID":"e2243778-8899-4f11-8805-91ea1f3f0b51","Type":"ContainerDied","Data":"d0bff9af0c2713ee3728761a3b394c6f592048f89f9286026ecd0b733d1bbcb9"}
Sep 06 05:00:32.451285 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.451261    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 05:00:32.538715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538613    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.538715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.538715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538691    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538813    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538833    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.538855    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzwjp\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp\") pod \"e2243778-8899-4f11-8805-91ea1f3f0b51\" (UID: \"e2243778-8899-4f11-8805-91ea1f3f0b51\") "
Sep 06 05:00:32.539262 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.539214    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:00:32.539772 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.539733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:00:32.541340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.541307    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:00:32.541519 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.541499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:00:32.541581 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.541509    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp" (OuterVolumeSpecName: "kube-api-access-jzwjp") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "kube-api-access-jzwjp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:00:32.541716 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.541684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:00:32.541716 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.541710    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:00:32.547485 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.547459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e2243778-8899-4f11-8805-91ea1f3f0b51" (UID: "e2243778-8899-4f11-8805-91ea1f3f0b51"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:00:32.640491 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640454    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-trusted-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640491 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640485    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2243778-8899-4f11-8805-91ea1f3f0b51-ca-trust-extracted\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640491 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640494    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-tls\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640503    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzwjp\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-kube-api-access-jzwjp\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640512    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2243778-8899-4f11-8805-91ea1f3f0b51-registry-certificates\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640520    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-image-registry-private-configuration\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640529    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2243778-8899-4f11-8805-91ea1f3f0b51-installation-pull-secrets\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:32.640694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:32.640538    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2243778-8899-4f11-8805-91ea1f3f0b51-bound-sa-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:00:33.432235 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:33.432199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d887db745-9w6xm" event={"ID":"e2243778-8899-4f11-8805-91ea1f3f0b51","Type":"ContainerDied","Data":"56707fa53bfc5ab44c4ef7ffcec47d86797df89da95b06a398b24f96abc45e1b"}
Sep 06 05:00:33.432681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:33.432264    2576 scope.go:117] "RemoveContainer" containerID="d0bff9af0c2713ee3728761a3b394c6f592048f89f9286026ecd0b733d1bbcb9"
Sep 06 05:00:33.432681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:33.432210    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d887db745-9w6xm"
Sep 06 05:00:33.452590 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:33.452569    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 05:00:33.455218 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:33.455198    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d887db745-9w6xm"]
Sep 06 05:00:34.384757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:34.384722    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" path="/var/lib/kubelet/pods/e2243778-8899-4f11-8805-91ea1f3f0b51/volumes"
Sep 06 05:00:36.406447 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:36.406411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v47hr"
Sep 06 05:00:38.321838 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:38.321816    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:00:38.322218 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:38.321856    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:00:38.330354 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:38.330335    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:00:38.330481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:38.330367    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:00:38.333832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:38.333814    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 06 05:00:39.379800 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.379769    2576 scope.go:117] "RemoveContainer" containerID="a881fdc9701dedb587c680d5b102e90f6780b43609c999293bd5cf1f605d47fd"
Sep 06 05:00:39.457584 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.457555    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:00:39.457771 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.457695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-t55rq" event={"ID":"45d3705d-364e-4846-b3d0-e10876db24d1","Type":"ContainerStarted","Data":"ec67552a169c5698ac32d9b2bbd1339ac374896604082ca0cfa62992486c9c86"}
Sep 06 05:00:39.458357 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.458107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 05:00:39.459524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.459501    2576 patch_prober.go:28] interesting pod/console-operator-85b454874b-t55rq container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.134.0.11:8443/readyz\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body=
Sep 06 05:00:39.459620 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.459541    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podUID="45d3705d-364e-4846-b3d0-e10876db24d1" containerName="console-operator" probeResult="failure" output="Get \"https://10.134.0.11:8443/readyz\": dial tcp 10.134.0.11:8443: connect: connection refused"
Sep 06 05:00:39.475018 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:39.474959    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-t55rq" podStartSLOduration=56.069551583 podStartE2EDuration="58.474939347s" totalImagesPullingTime="2.405387764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:00:39.474029898 +0000 UTC m=+301.563107591" watchObservedRunningTime="2026-09-06 05:00:39.474939347 +0000 UTC m=+301.564017045"
Sep 06 05:00:40.465777 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:40.465745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-t55rq"
Sep 06 05:00:42.380350 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:00:42.380312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 05:01:00.531921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:00.531873    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c23ef83-d706-4e78-92c7-2da117488b31" containerID="4f931e377b51e5c08ea1c32f0c9c67e46c14fe059f3b3f3e2acab6cee1082d52" exitCode=0
Sep 06 05:01:00.532350 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:00.531956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp" event={"ID":"1c23ef83-d706-4e78-92c7-2da117488b31","Type":"ContainerDied","Data":"4f931e377b51e5c08ea1c32f0c9c67e46c14fe059f3b3f3e2acab6cee1082d52"}
Sep 06 05:01:00.532350 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:00.532302    2576 scope.go:117] "RemoveContainer" containerID="4f931e377b51e5c08ea1c32f0c9c67e46c14fe059f3b3f3e2acab6cee1082d52"
Sep 06 05:01:01.537381 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:01.537347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjfnp" event={"ID":"1c23ef83-d706-4e78-92c7-2da117488b31","Type":"ContainerStarted","Data":"dfedf754a9c85ddbadc9955fc0035fd5132f2ed30bcceb4d2973ca5b661d351f"}
Sep 06 05:01:02.589547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:02.589504    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-kpp95_109e5587-86c1-487f-8166-f8e29284cfdf/cluster-monitoring-operator/0.log"
Sep 06 05:01:04.386508 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:04.386484    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/init-textfile/0.log"
Sep 06 05:01:04.587836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:04.587800    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/node-exporter/0.log"
Sep 06 05:01:04.787182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:04.787154    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/kube-rbac-proxy/0.log"
Sep 06 05:01:06.187636 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:06.187611    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/init-config-reloader/0.log"
Sep 06 05:01:06.390025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:06.389993    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/prometheus/0.log"
Sep 06 05:01:06.587241 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:06.587157    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/config-reloader/0.log"
Sep 06 05:01:06.788004 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:06.787976    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/thanos-sidecar/0.log"
Sep 06 05:01:06.987684 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:06.987652    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/kube-rbac-proxy-web/0.log"
Sep 06 05:01:07.187830 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:07.187801    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/kube-rbac-proxy/0.log"
Sep 06 05:01:07.391589 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:07.391500    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/kube-rbac-proxy-thanos/0.log"
Sep 06 05:01:10.190557 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:10.190528    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:01:10.390088 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:10.390053    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/3.log"
Sep 06 05:01:20.565140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:20.565107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:20.581151 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:20.581123    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:20.627677 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:20.627651    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:38.537672 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537589    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537880    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="prometheus" containerID="cri-o://f08b36a911cc6823eee394fc94c52d5bf32db210aab046284761fa8434c09d37" gracePeriod=600
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537937    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="config-reloader" containerID="cri-o://5c27f25a602dbc53073092b6a8d57954e95afe775d874571e5435da5f4d31f1a" gracePeriod=600
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537936    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy" containerID="cri-o://f30d868007262c5cf8a062ff2b652ba61831952acbbac7ab0afa2c071f4cbea4" gracePeriod=600
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537967    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-web" containerID="cri-o://87af86a275f6ab710587d8c3616fed2f87751ce09a64dcfd52caeaa408275761" gracePeriod=600
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.538011    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ba6b5c8cf6340553d9570fb71d89aa12d91644a293f1867fb03387fb899f9727" gracePeriod=600
Sep 06 05:01:38.538246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.537941    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="thanos-sidecar" containerID="cri-o://0964964d55eca420ffdbdf15edc1764509bc44db44cac07ac2802c33d262ae0c" gracePeriod=600
Sep 06 05:01:38.681370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681334    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="ba6b5c8cf6340553d9570fb71d89aa12d91644a293f1867fb03387fb899f9727" exitCode=0
Sep 06 05:01:38.681370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681366    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="f30d868007262c5cf8a062ff2b652ba61831952acbbac7ab0afa2c071f4cbea4" exitCode=0
Sep 06 05:01:38.681370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681379    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="87af86a275f6ab710587d8c3616fed2f87751ce09a64dcfd52caeaa408275761" exitCode=0
Sep 06 05:01:38.681370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681389    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="0964964d55eca420ffdbdf15edc1764509bc44db44cac07ac2802c33d262ae0c" exitCode=0
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681398    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="5c27f25a602dbc53073092b6a8d57954e95afe775d874571e5435da5f4d31f1a" exitCode=0
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681406    2576 generic.go:356] "Generic (PLEG): container finished" podID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerID="f08b36a911cc6823eee394fc94c52d5bf32db210aab046284761fa8434c09d37" exitCode=0
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"ba6b5c8cf6340553d9570fb71d89aa12d91644a293f1867fb03387fb899f9727"}
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"f30d868007262c5cf8a062ff2b652ba61831952acbbac7ab0afa2c071f4cbea4"}
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681604    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"87af86a275f6ab710587d8c3616fed2f87751ce09a64dcfd52caeaa408275761"}
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"0964964d55eca420ffdbdf15edc1764509bc44db44cac07ac2802c33d262ae0c"}
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681682    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"5c27f25a602dbc53073092b6a8d57954e95afe775d874571e5435da5f4d31f1a"}
Sep 06 05:01:38.681727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.681719    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"f08b36a911cc6823eee394fc94c52d5bf32db210aab046284761fa8434c09d37"}
Sep 06 05:01:38.782821 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.782798    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:38.934751 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934721    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.934927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.934927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934822    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.934927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934846    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.934927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.934927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934884    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934936    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.934969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935045    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935104    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jb9q\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935130    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935159    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935258    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935287    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935308    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db\") pod \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\" (UID: \"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf\") "
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935336    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935585    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935603    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:38.935815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.935721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.936360 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.936328    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.937032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.936998    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.937159 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.937035    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:01:38.937335 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.937309    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.938196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.938155    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.938311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.938213    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.938311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.938247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:01:38.939273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.939243    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:01:38.939554 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.939518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q" (OuterVolumeSpecName: "kube-api-access-8jb9q") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "kube-api-access-8jb9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:01:38.939554 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.939543    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config" (OuterVolumeSpecName: "config") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.939761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.939733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.939873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.939855    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.940455 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.940435    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.940517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.940471    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.940712 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.940693    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out" (OuterVolumeSpecName: "config-out") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:01:38.941359 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.941338    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:38.949365 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:38.949334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config" (OuterVolumeSpecName: "web-config") pod "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" (UID: "a78581b1-fe0a-4c52-9e03-1fccb9c3acbf"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 06 05:01:39.036840 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036796    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-grpc-tls\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.036840 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036837    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config-out\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.036840 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036846    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jb9q\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-kube-api-access-8jb9q\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036858    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-metrics-client-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036868    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036877    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036885    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036893    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-kube-rbac-proxy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036917    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036929    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036940    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-prometheus-k8s-db\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036952    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036963    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-web-config\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036973    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036983    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-tls-assets\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036990    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-configmap-metrics-client-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.036999    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.037060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.037010    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:01:39.688114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.688079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a78581b1-fe0a-4c52-9e03-1fccb9c3acbf","Type":"ContainerDied","Data":"7c3d2852e98165f3bfee7b1bb84c483baa02b5c0bfaf16f47a12b64bd38e8b64"}
Sep 06 05:01:39.688114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.688109    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.688647 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.688156    2576 scope.go:117] "RemoveContainer" containerID="ba6b5c8cf6340553d9570fb71d89aa12d91644a293f1867fb03387fb899f9727"
Sep 06 05:01:39.696034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.695810    2576 scope.go:117] "RemoveContainer" containerID="f30d868007262c5cf8a062ff2b652ba61831952acbbac7ab0afa2c071f4cbea4"
Sep 06 05:01:39.703519 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.703496    2576 scope.go:117] "RemoveContainer" containerID="87af86a275f6ab710587d8c3616fed2f87751ce09a64dcfd52caeaa408275761"
Sep 06 05:01:39.711810 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.711780    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:39.712460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.712445    2576 scope.go:117] "RemoveContainer" containerID="0964964d55eca420ffdbdf15edc1764509bc44db44cac07ac2802c33d262ae0c"
Sep 06 05:01:39.717095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.717067    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:39.719389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.719372    2576 scope.go:117] "RemoveContainer" containerID="5c27f25a602dbc53073092b6a8d57954e95afe775d874571e5435da5f4d31f1a"
Sep 06 05:01:39.725889 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.725874    2576 scope.go:117] "RemoveContainer" containerID="f08b36a911cc6823eee394fc94c52d5bf32db210aab046284761fa8434c09d37"
Sep 06 05:01:39.733080 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.733058    2576 scope.go:117] "RemoveContainer" containerID="6310d05cd348b5b1d249e993ac456ddc991878e2afd6e4399f18cb486854b58f"
Sep 06 05:01:39.740663 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740644    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:39.740890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740879    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="init-config-reloader"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740892    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="init-config-reloader"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740925    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="thanos-sidecar"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740932    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="thanos-sidecar"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740944    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740949    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy"
Sep 06 05:01:39.740954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740955    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-web"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740960    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-web"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740966    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740971    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740978    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-thanos"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740984    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-thanos"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740992    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="config-reloader"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.740996    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="config-reloader"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741001    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="prometheus"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741006    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="prometheus"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741045    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="config-reloader"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741053    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="thanos-sidecar"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741059    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741066    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2243778-8899-4f11-8805-91ea1f3f0b51" containerName="registry"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741072    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="prometheus"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741077    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-web"
Sep 06 05:01:39.741127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.741082    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" containerName="kube-rbac-proxy-thanos"
Sep 06 05:01:39.744530 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.744510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.747086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 06 05:01:39.747204 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747140    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 06 05:01:39.747481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747423    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 06 05:01:39.747481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747435    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 06 05:01:39.747481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747476    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h64vm\""
Sep 06 05:01:39.747761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747517    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 06 05:01:39.747761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3njndatvdmjim\""
Sep 06 05:01:39.747761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747476    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 06 05:01:39.747761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747727    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 06 05:01:39.748005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 06 05:01:39.748005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747804    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 06 05:01:39.748005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747805    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 06 05:01:39.748005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.747984    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 06 05:01:39.748212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.748153    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 06 05:01:39.748547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.748531    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 06 05:01:39.753121 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.753092    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 06 05:01:39.758086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.758063    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 06 05:01:39.763140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.763116    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:39.944448 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config-out\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944587    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.944655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944634    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwkfr\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-kube-api-access-nwkfr\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.944979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.945006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:39.945097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:39.945030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-web-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.045981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.045941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.045981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.045984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-web-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046249 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046249 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config-out\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwkfr\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-kube-api-access-nwkfr\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.046997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.046783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.047251 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.047118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.049606 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.049450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.049606 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.049501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.049859 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.049837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-web-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.050757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.050494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.050757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.050613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config-out\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.050757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.050637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.050757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.050694    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.050757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.050713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.051438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.051410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-config\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.051881 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.051856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.052510 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.052455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.053102 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.053084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.053409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.053387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.053456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.053387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.053774 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.053754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.054011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.053993    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.057849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.057826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwkfr\" (UniqueName: \"kubernetes.io/projected/68ec2c41-7ba2-43f9-8f9e-8b04288dec23-kube-api-access-nwkfr\") pod \"prometheus-k8s-0\" (UID: \"68ec2c41-7ba2-43f9-8f9e-8b04288dec23\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.059846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.059826    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:40.340331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.340303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 06 05:01:40.341214 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:01:40.341189    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ec2c41_7ba2_43f9_8f9e_8b04288dec23.slice/crio-e5058454858a4a3dbb2ca86e4919edde53badaf9e72c1fd367e2e2ba11c73b96 WatchSource:0}: Error finding container e5058454858a4a3dbb2ca86e4919edde53badaf9e72c1fd367e2e2ba11c73b96: Status 404 returned error can't find the container with id e5058454858a4a3dbb2ca86e4919edde53badaf9e72c1fd367e2e2ba11c73b96
Sep 06 05:01:40.385040 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.384999    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a78581b1-fe0a-4c52-9e03-1fccb9c3acbf" path="/var/lib/kubelet/pods/a78581b1-fe0a-4c52-9e03-1fccb9c3acbf/volumes"
Sep 06 05:01:40.695979 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.695948    2576 generic.go:356] "Generic (PLEG): container finished" podID="68ec2c41-7ba2-43f9-8f9e-8b04288dec23" containerID="352a190debcf3596025d97ba8421959c76acd11e1063503d8ee024d20cf03ed1" exitCode=0
Sep 06 05:01:40.696388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.696046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerDied","Data":"352a190debcf3596025d97ba8421959c76acd11e1063503d8ee024d20cf03ed1"}
Sep 06 05:01:40.696388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:40.696107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"e5058454858a4a3dbb2ca86e4919edde53badaf9e72c1fd367e2e2ba11c73b96"}
Sep 06 05:01:41.702674 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"c6c59cd4b2096cce24b24432af33ae5c402fd294651c1f12eff63e1c0fa66d46"}
Sep 06 05:01:41.703074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"aa69588e299e57e3a01107036818e685e2b9c67037c54e87e40ae81b6d426559"}
Sep 06 05:01:41.703074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"76fe0679994abc66305b42bf1212198bbef27f9ad425b3bafe1f0f4e3d7c75bb"}
Sep 06 05:01:41.703074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"b194f9f058beded9c1104b3996317c42a7e79c6d36662afdef4b03f1e7fe8ff6"}
Sep 06 05:01:41.703074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"07d4aac0a81032d278322f00a28299b05c2e3eccb3a8eee95370109a6f15ce89"}
Sep 06 05:01:41.703074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.702821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"68ec2c41-7ba2-43f9-8f9e-8b04288dec23","Type":"ContainerStarted","Data":"0db75a58f260a9d2768e29efbc071390a0494df9d357d4d107ac4d75c6eb34c7"}
Sep 06 05:01:41.735700 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:41.735643    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.718836278 podStartE2EDuration="2.735627705s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.791427ms" isStatefulPod=false podCreationTimestamp="2026-09-06 05:01:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:01:41.733807973 +0000 UTC m=+363.822885678" watchObservedRunningTime="2026-09-06 05:01:41.735627705 +0000 UTC m=+363.824705398"
Sep 06 05:01:45.060365 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:45.060331    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:01:52.343875 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.343819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 05:01:52.346316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.346295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f3f6321-5aaf-4656-ad4c-04109aeba114-metrics-certs\") pod \"network-metrics-daemon-r7t92\" (UID: \"4f3f6321-5aaf-4656-ad4c-04109aeba114\") " pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 05:01:52.584107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.584075    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b9kvk\""
Sep 06 05:01:52.591989 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.591968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r7t92"
Sep 06 05:01:52.861368 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.861318    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r7t92"]
Sep 06 05:01:52.863390 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:01:52.863358    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3f6321_5aaf_4656_ad4c_04109aeba114.slice/crio-a202c7a66139dfada7d7d2f16b101ca87a8f28798af76838afb0a004fe672514 WatchSource:0}: Error finding container a202c7a66139dfada7d7d2f16b101ca87a8f28798af76838afb0a004fe672514: Status 404 returned error can't find the container with id a202c7a66139dfada7d7d2f16b101ca87a8f28798af76838afb0a004fe672514
Sep 06 05:01:52.865372 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:52.865356    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:01:53.749851 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:53.749815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7t92" event={"ID":"4f3f6321-5aaf-4656-ad4c-04109aeba114","Type":"ContainerStarted","Data":"a202c7a66139dfada7d7d2f16b101ca87a8f28798af76838afb0a004fe672514"}
Sep 06 05:01:54.756404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:54.756359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7t92" event={"ID":"4f3f6321-5aaf-4656-ad4c-04109aeba114","Type":"ContainerStarted","Data":"49fc1a06e6a6d9d0d000162f00d4d233555820f87b971e28028cd9233010ff03"}
Sep 06 05:01:54.756798 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:54.756426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r7t92" event={"ID":"4f3f6321-5aaf-4656-ad4c-04109aeba114","Type":"ContainerStarted","Data":"9cef225c057d7b1d26800d907abf28854f2fbf61549bcd2ce51f266d867800f9"}
Sep 06 05:01:54.770880 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:01:54.770813    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r7t92" podStartSLOduration=374.813712681 podStartE2EDuration="6m15.770798562s" totalImagesPullingTime="957.085881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 04:55:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:01:54.76986168 +0000 UTC m=+376.858939378" watchObservedRunningTime="2026-09-06 05:01:54.770798562 +0000 UTC m=+376.859876255"
Sep 06 05:02:40.061032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:02:40.060988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:02:40.077650 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:02:40.077621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:02:40.936569 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:02:40.936541    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 06 05:03:17.407609 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.407566    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"]
Sep 06 05:03:17.411288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.411268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.413924 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.413885    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 06 05:03:17.414190 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.414076    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 06 05:03:17.414190 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.414180    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p86vv\""
Sep 06 05:03:17.414353 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.414238    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 06 05:03:17.414353 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.414277    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 06 05:03:17.422949 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.422924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"]
Sep 06 05:03:17.548655 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.548617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.548849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.548679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.548849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.548704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.548849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.548721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.548849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.548739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwr87\" (UniqueName: \"kubernetes.io/projected/73be6ee7-d026-426c-9054-8a94967d1e10-kube-api-access-jwr87\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.649970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.649887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.650176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.650016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.650176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.650040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.650176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.650068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.650176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.650120    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwr87\" (UniqueName: \"kubernetes.io/projected/73be6ee7-d026-426c-9054-8a94967d1e10-kube-api-access-jwr87\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.650504 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.650478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-manifests\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.652475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.652454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73be6ee7-d026-426c-9054-8a94967d1e10-tmp\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.652716 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.652699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-apiservice-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.652788 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.652771    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/73be6ee7-d026-426c-9054-8a94967d1e10-webhook-cert\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.666346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.666278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwr87\" (UniqueName: \"kubernetes.io/projected/73be6ee7-d026-426c-9054-8a94967d1e10-kube-api-access-jwr87\") pod \"opendatahub-operator-controller-manager-657c48fd97-2sjbp\" (UID: \"73be6ee7-d026-426c-9054-8a94967d1e10\") " pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.721407 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.721367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:17.998558 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:17.998531    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"]
Sep 06 05:03:18.000605 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:18.000576    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73be6ee7_d026_426c_9054_8a94967d1e10.slice/crio-e88c430cdc8e2f634f44921bc94c1c5da7432834bd699c122662817daf75cc87 WatchSource:0}: Error finding container e88c430cdc8e2f634f44921bc94c1c5da7432834bd699c122662817daf75cc87: Status 404 returned error can't find the container with id e88c430cdc8e2f634f44921bc94c1c5da7432834bd699c122662817daf75cc87
Sep 06 05:03:18.053684 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:18.053646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp" event={"ID":"73be6ee7-d026-426c-9054-8a94967d1e10","Type":"ContainerStarted","Data":"e88c430cdc8e2f634f44921bc94c1c5da7432834bd699c122662817daf75cc87"}
Sep 06 05:03:21.069563 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:21.069469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp" event={"ID":"73be6ee7-d026-426c-9054-8a94967d1e10","Type":"ContainerStarted","Data":"6ee94ffdb7afca3c23d1421f6b4dee23281197fdd64e5780d8a1ec92d694332f"}
Sep 06 05:03:22.075410 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:22.075372    2576 generic.go:356] "Generic (PLEG): container finished" podID="73be6ee7-d026-426c-9054-8a94967d1e10" containerID="6ee94ffdb7afca3c23d1421f6b4dee23281197fdd64e5780d8a1ec92d694332f" exitCode=0
Sep 06 05:03:22.075801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:22.075420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp" event={"ID":"73be6ee7-d026-426c-9054-8a94967d1e10","Type":"ContainerDied","Data":"6ee94ffdb7afca3c23d1421f6b4dee23281197fdd64e5780d8a1ec92d694332f"}
Sep 06 05:03:23.081870 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:23.081829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp" event={"ID":"73be6ee7-d026-426c-9054-8a94967d1e10","Type":"ContainerStarted","Data":"4e262acc21b96de51730844d1671d841cae7758708d6294077b62b28ceb14c2e"}
Sep 06 05:03:23.082378 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:23.081950    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:23.102643 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:23.102593    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp" podStartSLOduration=1.9705037939999999 podStartE2EDuration="6.102575657s" totalImagesPullingTime="3.130458502s" totalInitContainerRuntime="1.001613361s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:23.10101743 +0000 UTC m=+465.190095127" watchObservedRunningTime="2026-09-06 05:03:23.102575657 +0000 UTC m=+465.191653428"
Sep 06 05:03:34.089220 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:34.089189    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-657c48fd97-2sjbp"
Sep 06 05:03:35.315541 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.315511    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"]
Sep 06 05:03:35.321884 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.321862    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.325165 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.325139    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 06 05:03:35.325564 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.325546    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 06 05:03:35.325875 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.325860    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 06 05:03:35.326279 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.326261    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k6n4f\""
Sep 06 05:03:35.326621 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.326605    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 06 05:03:35.326992 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.326976    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 06 05:03:35.331304 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.331281    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"]
Sep 06 05:03:35.498337 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.498288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dea21342-0bed-44b8-b7d7-f1c1b70221f4-manager-config\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.498552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.498359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-metrics-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.498552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.498488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zbwc\" (UniqueName: \"kubernetes.io/projected/dea21342-0bed-44b8-b7d7-f1c1b70221f4-kube-api-access-4zbwc\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.498552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.498529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.599586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.599488    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zbwc\" (UniqueName: \"kubernetes.io/projected/dea21342-0bed-44b8-b7d7-f1c1b70221f4-kube-api-access-4zbwc\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.599586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.599529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.599586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.599574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dea21342-0bed-44b8-b7d7-f1c1b70221f4-manager-config\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.599879 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.599617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-metrics-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.600298 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.600275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dea21342-0bed-44b8-b7d7-f1c1b70221f4-manager-config\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.602396 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.602375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.602481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.602442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dea21342-0bed-44b8-b7d7-f1c1b70221f4-metrics-cert\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.615336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.615310    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zbwc\" (UniqueName: \"kubernetes.io/projected/dea21342-0bed-44b8-b7d7-f1c1b70221f4-kube-api-access-4zbwc\") pod \"lws-controller-manager-7d8d45985c-vrr6q\" (UID: \"dea21342-0bed-44b8-b7d7-f1c1b70221f4\") " pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.637644 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.637614    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:35.920009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:35.919924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"]
Sep 06 05:03:35.924621 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:35.924574    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddea21342_0bed_44b8_b7d7_f1c1b70221f4.slice/crio-8074465cc84b7e81e8d46cc10cbc994820ac96b89f2671010840a96905f8ef35 WatchSource:0}: Error finding container 8074465cc84b7e81e8d46cc10cbc994820ac96b89f2671010840a96905f8ef35: Status 404 returned error can't find the container with id 8074465cc84b7e81e8d46cc10cbc994820ac96b89f2671010840a96905f8ef35
Sep 06 05:03:36.131139 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:36.131101    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q" event={"ID":"dea21342-0bed-44b8-b7d7-f1c1b70221f4","Type":"ContainerStarted","Data":"8074465cc84b7e81e8d46cc10cbc994820ac96b89f2671010840a96905f8ef35"}
Sep 06 05:03:39.146594 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:39.146546    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q" event={"ID":"dea21342-0bed-44b8-b7d7-f1c1b70221f4","Type":"ContainerStarted","Data":"34e983d137047d534f306ff116861758b7dc52a85942c022c01518da32e6a246"}
Sep 06 05:03:39.147030 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:39.146671    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:39.161971 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:39.161891    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q" podStartSLOduration=1.540248579 podStartE2EDuration="4.161874118s" totalImagesPullingTime="2.621625539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:39.161201956 +0000 UTC m=+481.250279650" watchObservedRunningTime="2026-09-06 05:03:39.161874118 +0000 UTC m=+481.250951812"
Sep 06 05:03:41.581967 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.581926    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:41.584512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.584490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.587132 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.587110    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 06 05:03:41.587258 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.587216    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lzdvg\""
Sep 06 05:03:41.594815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.594786    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:41.756739 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.756707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.756961 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.756748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.756961 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.756770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.756961 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.756880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.757106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.756966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.757106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.757005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.757106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.757041    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pwnb\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.757106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.757063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.757106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.757089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.857695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.857695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.857695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857747    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pwnb\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.857917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.858204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.858283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858450 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.858342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858450 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.858398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.858633 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.858613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.860659 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.860638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.860832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.860813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.881980 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.881944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.882148 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.882134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pwnb\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcplx28\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:41.896002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:41.895972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:42.177968 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:42.177937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:42.180912 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:42.180856    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c8c542_bbc3_4a6a_8965_63c0dd7c6cb0.slice/crio-93cce51a4d95a621e4131aef7f1f22dcb10dcd5c89077c061761dd1c30e109a1 WatchSource:0}: Error finding container 93cce51a4d95a621e4131aef7f1f22dcb10dcd5c89077c061761dd1c30e109a1: Status 404 returned error can't find the container with id 93cce51a4d95a621e4131aef7f1f22dcb10dcd5c89077c061761dd1c30e109a1
Sep 06 05:03:43.165174 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:43.165132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" event={"ID":"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0","Type":"ContainerStarted","Data":"93cce51a4d95a621e4131aef7f1f22dcb10dcd5c89077c061761dd1c30e109a1"}
Sep 06 05:03:46.362851 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:46.362811    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:03:46.363145 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:46.362888    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:03:46.363145 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:46.362941    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:03:47.055106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.055065    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q6txt"]
Sep 06 05:03:47.058115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.058090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.060968 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.060944    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 06 05:03:47.061230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.061205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xkp98\""
Sep 06 05:03:47.071334 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.071303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q6txt"]
Sep 06 05:03:47.104331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.104293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.104524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.104359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhzcg\" (UniqueName: \"kubernetes.io/projected/5f0ddd00-140c-4fbe-bb49-9957077cbcce-kube-api-access-zhzcg\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.184992 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.184951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" event={"ID":"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0","Type":"ContainerStarted","Data":"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"}
Sep 06 05:03:47.204886 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.204824    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" podStartSLOduration=2.025810549 podStartE2EDuration="6.204805932s" totalImagesPullingTime="4.178995383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:47.20325394 +0000 UTC m=+489.292331657" watchObservedRunningTime="2026-09-06 05:03:47.204805932 +0000 UTC m=+489.293883625"
Sep 06 05:03:47.205084 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.205024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.205207 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:03:47.205182    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 06 05:03:47.205441 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.205224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhzcg\" (UniqueName: \"kubernetes.io/projected/5f0ddd00-140c-4fbe-bb49-9957077cbcce-kube-api-access-zhzcg\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.205441 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:03:47.205425    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert podName:5f0ddd00-140c-4fbe-bb49-9957077cbcce nodeName:}" failed. No retries permitted until 2026-09-06 05:03:47.70540449 +0000 UTC m=+489.794482160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert") pod "odh-model-controller-7998c67dc4-q6txt" (UID: "5f0ddd00-140c-4fbe-bb49-9957077cbcce") : secret "odh-model-controller-webhook-cert" not found
Sep 06 05:03:47.217275 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.217238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhzcg\" (UniqueName: \"kubernetes.io/projected/5f0ddd00-140c-4fbe-bb49-9957077cbcce-kube-api-access-zhzcg\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.639329 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.639289    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7vmbq"]
Sep 06 05:03:47.641703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.641685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.644221 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.644191    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 06 05:03:47.644357 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.644239    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-ls42m\""
Sep 06 05:03:47.653374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.653352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7vmbq"]
Sep 06 05:03:47.708919 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.708876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.709120 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.708952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd2vm\" (UniqueName: \"kubernetes.io/projected/2153bc10-8b55-40ae-9c7e-154f1b917990-kube-api-access-sd2vm\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.709120 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.709010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.711640 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.711614    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f0ddd00-140c-4fbe-bb49-9957077cbcce-cert\") pod \"odh-model-controller-7998c67dc4-q6txt\" (UID: \"5f0ddd00-140c-4fbe-bb49-9957077cbcce\") " pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:47.810242 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.810200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.810242 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.810243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd2vm\" (UniqueName: \"kubernetes.io/projected/2153bc10-8b55-40ae-9c7e-154f1b917990-kube-api-access-sd2vm\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.810503 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:03:47.810387    2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 06 05:03:47.810503 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:03:47.810466    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs podName:2153bc10-8b55-40ae-9c7e-154f1b917990 nodeName:}" failed. No retries permitted until 2026-09-06 05:03:48.310448218 +0000 UTC m=+490.399525892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs") pod "model-serving-api-54d89bd5fc-7vmbq" (UID: "2153bc10-8b55-40ae-9c7e-154f1b917990") : secret "model-serving-api-tls" not found
Sep 06 05:03:47.818850 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.818822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd2vm\" (UniqueName: \"kubernetes.io/projected/2153bc10-8b55-40ae-9c7e-154f1b917990-kube-api-access-sd2vm\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:47.896863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.896772    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:47.898478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.898442    2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcplx28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Sep 06 05:03:47.898636 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.898500    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Sep 06 05:03:47.967627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:47.967594    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:48.195211 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.195175    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:48.242285 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.242261    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q6txt"]
Sep 06 05:03:48.244663 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:48.244635    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0ddd00_140c_4fbe_bb49_9957077cbcce.slice/crio-8da3a6ebabfaf3640551bb45dc32b53d692d1558a88ff103c93c355bca65e7e0 WatchSource:0}: Error finding container 8da3a6ebabfaf3640551bb45dc32b53d692d1558a88ff103c93c355bca65e7e0: Status 404 returned error can't find the container with id 8da3a6ebabfaf3640551bb45dc32b53d692d1558a88ff103c93c355bca65e7e0
Sep 06 05:03:48.316032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.315993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:48.318692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.318664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2153bc10-8b55-40ae-9c7e-154f1b917990-tls-certs\") pod \"model-serving-api-54d89bd5fc-7vmbq\" (UID: \"2153bc10-8b55-40ae-9c7e-154f1b917990\") " pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:48.552230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.552140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:48.827393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.827365    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7vmbq"]
Sep 06 05:03:48.829674 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:48.829641    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2153bc10_8b55_40ae_9c7e_154f1b917990.slice/crio-9b6432812e8674452b0c9369802624117ed6388d86a11f9cce7dc4bc648b52e0 WatchSource:0}: Error finding container 9b6432812e8674452b0c9369802624117ed6388d86a11f9cce7dc4bc648b52e0: Status 404 returned error can't find the container with id 9b6432812e8674452b0c9369802624117ed6388d86a11f9cce7dc4bc648b52e0
Sep 06 05:03:48.897324 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.897291    2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcplx28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Sep 06 05:03:48.897573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:48.897349    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Sep 06 05:03:49.196677 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:49.196638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq" event={"ID":"2153bc10-8b55-40ae-9c7e-154f1b917990","Type":"ContainerStarted","Data":"9b6432812e8674452b0c9369802624117ed6388d86a11f9cce7dc4bc648b52e0"}
Sep 06 05:03:49.198065 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:49.198028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt" event={"ID":"5f0ddd00-140c-4fbe-bb49-9957077cbcce","Type":"ContainerStarted","Data":"8da3a6ebabfaf3640551bb45dc32b53d692d1558a88ff103c93c355bca65e7e0"}
Sep 06 05:03:49.896886 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:49.896841    2576 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcplx28 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Sep 06 05:03:49.897386 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:49.896938    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Sep 06 05:03:50.152800 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:50.152721    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7d8d45985c-vrr6q"
Sep 06 05:03:50.203964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:50.203922    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy" containerID="cri-o://06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98" gracePeriod=30
Sep 06 05:03:53.219940 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.219884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq" event={"ID":"2153bc10-8b55-40ae-9c7e-154f1b917990","Type":"ContainerStarted","Data":"2f9db250f790b773e8ade257047441745c9cc94f917f918a4c6c5c6d4d728305"}
Sep 06 05:03:53.220388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.220086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:03:53.221497 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.221476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt" event={"ID":"5f0ddd00-140c-4fbe-bb49-9957077cbcce","Type":"ContainerStarted","Data":"cabbb91584acb8948268d8cc3f147d8ebeae7c0ef2986b1a7da3afeafce40f56"}
Sep 06 05:03:53.221581 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.221562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:03:53.236103 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.236043    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq" podStartSLOduration=2.5965974149999997 podStartE2EDuration="6.236026756s" totalImagesPullingTime="3.639429341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:53.235116713 +0000 UTC m=+495.324194404" watchObservedRunningTime="2026-09-06 05:03:53.236026756 +0000 UTC m=+495.325104444"
Sep 06 05:03:53.250176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:53.250119    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt" podStartSLOduration=2.034676679 podStartE2EDuration="6.250104214s" totalImagesPullingTime="4.215427535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:53.249827602 +0000 UTC m=+495.338905296" watchObservedRunningTime="2026-09-06 05:03:53.250104214 +0000 UTC m=+495.339181907"
Sep 06 05:03:54.078740 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.078705    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"]
Sep 06 05:03:54.081107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.081091    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.083651 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.083622    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 06 05:03:54.083781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.083664    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 06 05:03:54.083781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.083667    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 06 05:03:54.083781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.083689    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qpw86\""
Sep 06 05:03:54.094709 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.094686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"]
Sep 06 05:03:54.173795 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173752    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b65gd\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-kube-api-access-b65gd\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.173795 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.174074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.174074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.174074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.174074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96182306-f539-43a9-9a55-aca1aa258667-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.174074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.173984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.274893 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.274858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96182306-f539-43a9-9a55-aca1aa258667-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b65gd\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-kube-api-access-b65gd\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.275405 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.275327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.276444 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.276405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.277695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.277672    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.277805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.277712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96182306-f539-43a9-9a55-aca1aa258667-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.277805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.277795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.277940 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.277848    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96182306-f539-43a9-9a55-aca1aa258667-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.282946 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.282926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.283114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.283096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b65gd\" (UniqueName: \"kubernetes.io/projected/96182306-f539-43a9-9a55-aca1aa258667-kube-api-access-b65gd\") pod \"istiod-openshift-gateway-5677ccb49b-h74bm\" (UID: \"96182306-f539-43a9-9a55-aca1aa258667\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.390711 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.390622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:54.664392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:54.664313    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"]
Sep 06 05:03:54.666615 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:03:54.666583    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96182306_f539_43a9_9a55_aca1aa258667.slice/crio-cbbdf431769543e43afe0984e630634941b7627399700562e18bbc4d74b0a8d6 WatchSource:0}: Error finding container cbbdf431769543e43afe0984e630634941b7627399700562e18bbc4d74b0a8d6: Status 404 returned error can't find the container with id cbbdf431769543e43afe0984e630634941b7627399700562e18bbc4d74b0a8d6
Sep 06 05:03:55.230739 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.230709    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm" event={"ID":"96182306-f539-43a9-9a55-aca1aa258667","Type":"ContainerStarted","Data":"cbbdf431769543e43afe0984e630634941b7627399700562e18bbc4d74b0a8d6"}
Sep 06 05:03:55.467441 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.467415    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:55.587726 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587631    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.587726 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587676    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.587726 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587705    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pwnb\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587786    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587820    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587973    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.587993    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:03:55.588045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588009    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert\") pod \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\" (UID: \"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0\") "
Sep 06 05:03:55.588406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:03:55.588406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588229    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:03:55.588406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588374    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:03:55.588406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588374    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:03:55.588406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588384    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.588665 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588422    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.588665 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.588437    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.590553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.590520    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb" (OuterVolumeSpecName: "kube-api-access-5pwnb") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "kube-api-access-5pwnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:03:55.590553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.590526    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:03:55.590752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.590737    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:03:55.590857 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.590759    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" (UID: "a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:03:55.688874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688833    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.688874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688867    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.688874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688876    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pwnb\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-kube-api-access-5pwnb\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.689135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688886    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.689135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688895    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:55.689135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:55.688921    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:03:56.237875 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.237811    2576 generic.go:356] "Generic (PLEG): container finished" podID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerID="06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98" exitCode=0
Sep 06 05:03:56.238069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.237885    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"
Sep 06 05:03:56.238069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.237913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" event={"ID":"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0","Type":"ContainerDied","Data":"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"}
Sep 06 05:03:56.238069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.237978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28" event={"ID":"a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0","Type":"ContainerDied","Data":"93cce51a4d95a621e4131aef7f1f22dcb10dcd5c89077c061761dd1c30e109a1"}
Sep 06 05:03:56.238069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.238007    2576 scope.go:117] "RemoveContainer" containerID="06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"
Sep 06 05:03:56.249430 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.249408    2576 scope.go:117] "RemoveContainer" containerID="06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"
Sep 06 05:03:56.249885 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:03:56.249854    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98\": container with ID starting with 06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98 not found: ID does not exist" containerID="06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"
Sep 06 05:03:56.249988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.249895    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98"} err="failed to get container status \"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98\": rpc error: code = NotFound desc = could not find container \"06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98\": container with ID starting with 06c50ce5ae1f89e5aac2aabb70844d22f348c04e51453f5ba5bb24b61fc64e98 not found: ID does not exist"
Sep 06 05:03:56.262573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.262541    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:56.266511 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.266483    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcplx28"]
Sep 06 05:03:56.387068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:56.387029    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" path="/var/lib/kubelet/pods/a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0/volumes"
Sep 06 05:03:57.569104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:57.569064    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:03:57.569409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:57.569132    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:03:58.252394 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:58.252258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm" event={"ID":"96182306-f539-43a9-9a55-aca1aa258667","Type":"ContainerStarted","Data":"7bc8bf51613e38124c65167fa62343ef07d5e2110721575e95de65bbd6b5b21a"}
Sep 06 05:03:58.252962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:58.252938    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:58.254752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:58.254709    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-h74bm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:03:58.254872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:58.254777    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm" podUID="96182306-f539-43a9-9a55-aca1aa258667" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:03:59.257157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:59.257123    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm"
Sep 06 05:03:59.282793 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:03:59.282726    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-h74bm" podStartSLOduration=2.382738153 podStartE2EDuration="5.282710406s" totalImagesPullingTime="2.899972253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:03:58.286556988 +0000 UTC m=+500.375634687" watchObservedRunningTime="2026-09-06 05:03:59.282710406 +0000 UTC m=+501.371788099"
Sep 06 05:04:04.227378 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:04:04.227338    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-q6txt"
Sep 06 05:04:04.229618 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:04:04.229592    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7vmbq"
Sep 06 05:05:38.348297 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:05:38.348268    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:05:38.348852 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:05:38.348421    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:05:38.355743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:05:38.355722    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:05:38.355861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:05:38.355774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:06:36.601146 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.601081    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"]
Sep 06 05:06:36.601986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.601963    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy"
Sep 06 05:06:36.601986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.601984    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy"
Sep 06 05:06:36.602158 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.602041    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1c8c542-bbc3-4a6a-8965-63c0dd7c6cb0" containerName="istio-proxy"
Sep 06 05:06:36.604231 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.604199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.606757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.606722    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4qvb6\""
Sep 06 05:06:36.606874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.606808    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 06 05:06:36.607843 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.607819    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 06 05:06:36.607989 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.607881    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 06 05:06:36.612965 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.612941    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"]
Sep 06 05:06:36.695782 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.695744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpvw4\" (UniqueName: \"kubernetes.io/projected/24318767-9f33-4bbd-8bb2-f5022018de8b-kube-api-access-wpvw4\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.695998 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.695828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/24318767-9f33-4bbd-8bb2-f5022018de8b-data\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.797087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.797051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpvw4\" (UniqueName: \"kubernetes.io/projected/24318767-9f33-4bbd-8bb2-f5022018de8b-kube-api-access-wpvw4\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.797284 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.797124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/24318767-9f33-4bbd-8bb2-f5022018de8b-data\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.797515 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.797495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/24318767-9f33-4bbd-8bb2-f5022018de8b-data\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.805401 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.805369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpvw4\" (UniqueName: \"kubernetes.io/projected/24318767-9f33-4bbd-8bb2-f5022018de8b-kube-api-access-wpvw4\") pod \"postgres-69fb4594cf-hz7r6\" (UID: \"24318767-9f33-4bbd-8bb2-f5022018de8b\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:36.915195 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:36.915098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:37.185044 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:37.185011    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"]
Sep 06 05:06:37.188107 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:06:37.188066    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24318767_9f33_4bbd_8bb2_f5022018de8b.slice/crio-6cb60759ec15b228f88d0fbf021e8646e2f9de6d243fa65fc89cb58e47dfae8a WatchSource:0}: Error finding container 6cb60759ec15b228f88d0fbf021e8646e2f9de6d243fa65fc89cb58e47dfae8a: Status 404 returned error can't find the container with id 6cb60759ec15b228f88d0fbf021e8646e2f9de6d243fa65fc89cb58e47dfae8a
Sep 06 05:06:37.940239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:37.940187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6" event={"ID":"24318767-9f33-4bbd-8bb2-f5022018de8b","Type":"ContainerStarted","Data":"6cb60759ec15b228f88d0fbf021e8646e2f9de6d243fa65fc89cb58e47dfae8a"}
Sep 06 05:06:41.703103 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:41.703071    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 06 05:06:41.962766 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:41.962673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6" event={"ID":"24318767-9f33-4bbd-8bb2-f5022018de8b","Type":"ContainerStarted","Data":"0880ee369184c3bd6ba90d918c9d6ed0a66f3921a6ad31a267923ca36d643dae"}
Sep 06 05:06:41.962964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:41.962787    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:06:41.978965 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:41.978893    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6" podStartSLOduration=1.468253837 podStartE2EDuration="5.978878105s" totalImagesPullingTime="4.510624268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:06:41.976915018 +0000 UTC m=+664.065992727" watchObservedRunningTime="2026-09-06 05:06:41.978878105 +0000 UTC m=+664.067955799"
Sep 06 05:06:47.997133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:06:47.997103    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hz7r6"
Sep 06 05:07:32.291413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.291333    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"]
Sep 06 05:07:32.294593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.294570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.297346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.297316    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2ntxn\""
Sep 06 05:07:32.297481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.297409    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 06 05:07:32.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.305950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"]
Sep 06 05:07:32.394396 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.394360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af58991f-5113-45a2-879e-2baf61e00a52-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.394590 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.394459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgt7j\" (UniqueName: \"kubernetes.io/projected/af58991f-5113-45a2-879e-2baf61e00a52-kube-api-access-rgt7j\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.495803 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.495758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af58991f-5113-45a2-879e-2baf61e00a52-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.495998 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.495846    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgt7j\" (UniqueName: \"kubernetes.io/projected/af58991f-5113-45a2-879e-2baf61e00a52-kube-api-access-rgt7j\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.496413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.496395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af58991f-5113-45a2-879e-2baf61e00a52-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.507039 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.507006    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgt7j\" (UniqueName: \"kubernetes.io/projected/af58991f-5113-45a2-879e-2baf61e00a52-kube-api-access-rgt7j\") pod \"payload-pre-processing-59fbc47bf8-cfhdf\" (UID: \"af58991f-5113-45a2-879e-2baf61e00a52\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.605352 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.605257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:32.875208 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.875122    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"]
Sep 06 05:07:32.878327 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:07:32.878302    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf58991f_5113_45a2_879e_2baf61e00a52.slice/crio-8ba01c51498eb13322ec5b827ddef0e7c9a5d94edb6006353de6a7cceb5e179d WatchSource:0}: Error finding container 8ba01c51498eb13322ec5b827ddef0e7c9a5d94edb6006353de6a7cceb5e179d: Status 404 returned error can't find the container with id 8ba01c51498eb13322ec5b827ddef0e7c9a5d94edb6006353de6a7cceb5e179d
Sep 06 05:07:32.880259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:32.880241    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 06 05:07:33.179569 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:33.179537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf" event={"ID":"af58991f-5113-45a2-879e-2baf61e00a52","Type":"ContainerStarted","Data":"8ba01c51498eb13322ec5b827ddef0e7c9a5d94edb6006353de6a7cceb5e179d"}
Sep 06 05:07:36.196079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:36.196040    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf" event={"ID":"af58991f-5113-45a2-879e-2baf61e00a52","Type":"ContainerStarted","Data":"c74c4a84538f5ca81193883fb8963a30c5003254b325176dc65835f28834bb8e"}
Sep 06 05:07:36.196507 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:36.196201    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:36.212433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:36.212387    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf" podStartSLOduration=1.847093429 podStartE2EDuration="4.21237255s" totalImagesPullingTime="2.365279121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:36.21101252 +0000 UTC m=+718.300090214" watchObservedRunningTime="2026-09-06 05:07:36.21237255 +0000 UTC m=+718.301450243"
Sep 06 05:07:47.204142 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:47.204107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cfhdf"
Sep 06 05:07:50.543225 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.543191    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"]
Sep 06 05:07:50.549202 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.549168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.551578 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.551554    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 06 05:07:50.551735 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.551610    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 06 05:07:50.552715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.552656    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9srnm\""
Sep 06 05:07:50.552860 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.552656    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 06 05:07:50.555690 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.555668    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"]
Sep 06 05:07:50.664775 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.664730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.664775 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.664779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.665029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.664858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.665029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.664939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94m7c\" (UniqueName: \"kubernetes.io/projected/519d5c9c-5052-44e9-8bc3-3a39af35ab73-kube-api-access-94m7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.665029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.664970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.665029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.665006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94m7c\" (UniqueName: \"kubernetes.io/projected/519d5c9c-5052-44e9-8bc3-3a39af35ab73-kube-api-access-94m7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765971 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.765971 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.765955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.766213 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.766183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.766344 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.766323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.766399 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.766350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.768183 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.768160    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/519d5c9c-5052-44e9-8bc3-3a39af35ab73-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.768288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.768199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/519d5c9c-5052-44e9-8bc3-3a39af35ab73-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.773221 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.773200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94m7c\" (UniqueName: \"kubernetes.io/projected/519d5c9c-5052-44e9-8bc3-3a39af35ab73-kube-api-access-94m7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5\" (UID: \"519d5c9c-5052-44e9-8bc3-3a39af35ab73\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:50.861239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:50.861151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:07:51.132796 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:51.132769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"]
Sep 06 05:07:51.267264 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:51.267229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5" event={"ID":"519d5c9c-5052-44e9-8bc3-3a39af35ab73","Type":"ContainerStarted","Data":"b554f0fbe79678e899641257230ee7c5675b6e9cd2edb68715c8086224551d4a"}
Sep 06 05:07:52.273159 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:52.273126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5" event={"ID":"519d5c9c-5052-44e9-8bc3-3a39af35ab73","Type":"ContainerStarted","Data":"4edd7194496c98b69995a7a74b8d1f9b37fb7f729997af70368eada3e1cc67a6"}
Sep 06 05:07:53.294438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:07:53.294382    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5" podStartSLOduration=2.225400552 podStartE2EDuration="3.294361924s" totalImagesPullingTime="1.068961372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:07:53.292475041 +0000 UTC m=+735.381552769" watchObservedRunningTime="2026-09-06 05:07:53.294361924 +0000 UTC m=+735.383439630"
Sep 06 05:08:00.861689 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:00.861648    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:08:00.862145 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:00.861714    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:08:00.874611 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:00.874590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:08:01.322587 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:01.322560    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5"
Sep 06 05:08:12.042387 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.042353    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"]
Sep 06 05:08:12.046065 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.046043    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.048556 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.048533    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 06 05:08:12.057951 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.057887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"]
Sep 06 05:08:12.158964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.158892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.159131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.158973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeb179a-b7aa-4d4d-985b-fb1616129323-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.159131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.159034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw6xm\" (UniqueName: \"kubernetes.io/projected/bdeb179a-b7aa-4d4d-985b-fb1616129323-kube-api-access-fw6xm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.159131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.159079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.159131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.159100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.159273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.159142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260226 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260226 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeb179a-b7aa-4d4d-985b-fb1616129323-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw6xm\" (UniqueName: \"kubernetes.io/projected/bdeb179a-b7aa-4d4d-985b-fb1616129323-kube-api-access-fw6xm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.260681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.260671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.262850 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.262827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdeb179a-b7aa-4d4d-985b-fb1616129323-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.262976 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.262919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdeb179a-b7aa-4d4d-985b-fb1616129323-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.268553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.268530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw6xm\" (UniqueName: \"kubernetes.io/projected/bdeb179a-b7aa-4d4d-985b-fb1616129323-kube-api-access-fw6xm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7nk25\" (UID: \"bdeb179a-b7aa-4d4d-985b-fb1616129323\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.356594 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.356505    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:12.625808 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:12.625726    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"]
Sep 06 05:08:12.628888 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:08:12.628859    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdeb179a_b7aa_4d4d_985b_fb1616129323.slice/crio-217c9d0b3585a024b813f47fe37e103c95a3b51431a3a62a205af7ff04b700a3 WatchSource:0}: Error finding container 217c9d0b3585a024b813f47fe37e103c95a3b51431a3a62a205af7ff04b700a3: Status 404 returned error can't find the container with id 217c9d0b3585a024b813f47fe37e103c95a3b51431a3a62a205af7ff04b700a3
Sep 06 05:08:13.363132 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:13.363098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25" event={"ID":"bdeb179a-b7aa-4d4d-985b-fb1616129323","Type":"ContainerStarted","Data":"2687788a33534904698ae729830a965caa988131859c4a1c31ca57051eac378c"}
Sep 06 05:08:13.363516 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:13.363144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25" event={"ID":"bdeb179a-b7aa-4d4d-985b-fb1616129323","Type":"ContainerStarted","Data":"217c9d0b3585a024b813f47fe37e103c95a3b51431a3a62a205af7ff04b700a3"}
Sep 06 05:08:13.380849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:13.380792    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25" podStartSLOduration=1.219636227 podStartE2EDuration="1.380775362s" totalImagesPullingTime="161.139135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:08:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:08:13.379566233 +0000 UTC m=+755.468643939" watchObservedRunningTime="2026-09-06 05:08:13.380775362 +0000 UTC m=+755.469853086"
Sep 06 05:08:22.356797 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:22.356761    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:22.357241 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:22.356827    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:22.369349 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:22.369326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:22.417251 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:22.417223    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7nk25"
Sep 06 05:08:45.241967 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.241935    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"]
Sep 06 05:08:45.245832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.245816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.248480 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.248457    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 06 05:08:45.257490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.257468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"]
Sep 06 05:08:45.353325 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.353325 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.353545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353369    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.353545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.353545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522248f3-a4fa-461d-8649-a2f07b110152-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.353545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.353476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvbl6\" (UniqueName: \"kubernetes.io/projected/522248f3-a4fa-461d-8649-a2f07b110152-kube-api-access-pvbl6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454740 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522248f3-a4fa-461d-8649-a2f07b110152-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvbl6\" (UniqueName: \"kubernetes.io/projected/522248f3-a4fa-461d-8649-a2f07b110152-kube-api-access-pvbl6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.454973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.454959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.455254 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.455212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.455311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.455250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.455355 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.455311    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.457417 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.457392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/522248f3-a4fa-461d-8649-a2f07b110152-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.457525 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.457508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/522248f3-a4fa-461d-8649-a2f07b110152-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.463130 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.463106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvbl6\" (UniqueName: \"kubernetes.io/projected/522248f3-a4fa-461d-8649-a2f07b110152-kube-api-access-pvbl6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl\" (UID: \"522248f3-a4fa-461d-8649-a2f07b110152\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.556324 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.556226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:45.827430 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:45.827409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"]
Sep 06 05:08:45.829955 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:08:45.829928    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522248f3_a4fa_461d_8649_a2f07b110152.slice/crio-e9313fd2771d60318f8dbdf86e13a333b35e729f8cda16d08f7d2733ec65953e WatchSource:0}: Error finding container e9313fd2771d60318f8dbdf86e13a333b35e729f8cda16d08f7d2733ec65953e: Status 404 returned error can't find the container with id e9313fd2771d60318f8dbdf86e13a333b35e729f8cda16d08f7d2733ec65953e
Sep 06 05:08:46.509067 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:46.509034    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl" event={"ID":"522248f3-a4fa-461d-8649-a2f07b110152","Type":"ContainerStarted","Data":"be92c09dd99caa95bab42bb451584f2511abd7a0c78978c6f19baa34e51d4721"}
Sep 06 05:08:46.509448 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:46.509080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl" event={"ID":"522248f3-a4fa-461d-8649-a2f07b110152","Type":"ContainerStarted","Data":"e9313fd2771d60318f8dbdf86e13a333b35e729f8cda16d08f7d2733ec65953e"}
Sep 06 05:08:46.525009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:46.524885    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl" podStartSLOduration=1.362474925 podStartE2EDuration="1.524863658s" totalImagesPullingTime="162.388733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:08:46.524492656 +0000 UTC m=+788.613570383" watchObservedRunningTime="2026-09-06 05:08:46.524863658 +0000 UTC m=+788.613941352"
Sep 06 05:08:55.556591 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:55.556552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:55.557045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:55.556617    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:55.569549 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:55.569524    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:08:56.564972 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:08:56.564936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl"
Sep 06 05:10:38.375207 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:38.375124    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:10:38.376687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:38.376656    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:10:38.384079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:38.384061    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:10:38.385533 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:38.385511    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:10:45.993621 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:45.993589    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:10:45.997125 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:45.997103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.000037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.000018    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-d3a1cb\""
Sep 06 05:10:46.000144 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.000081    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-d3a1cb-dockercfg-2j6sn\""
Sep 06 05:10:46.007119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.007092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:10:46.029308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.029279    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:10:46.032718 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.032701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.042675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.042648    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:10:46.076815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.076782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.077017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.076854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v45rv\" (UniqueName: \"kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.077017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.076995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.077135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.077112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b76sx\" (UniqueName: \"kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.177788 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.177751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b76sx\" (UniqueName: \"kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.177788 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.177793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.178092 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.177834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v45rv\" (UniqueName: \"kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.178092 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.177889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.178781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.178757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.178846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.178757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.185517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.185489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b76sx\" (UniqueName: \"kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx\") pod \"payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.185635 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.185493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v45rv\" (UniqueName: \"kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv\") pod \"payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.308133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.308045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:46.344541 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.344504    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:46.586683 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.586602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:10:46.590101 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:46.590074    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a82f33_a549_46e8_8272_6b82f4ef1b82.slice/crio-57caebf9d4be4e71da611bcdb4ff1e9e98a44a56df4b95a556c6e9462bd3da4a WatchSource:0}: Error finding container 57caebf9d4be4e71da611bcdb4ff1e9e98a44a56df4b95a556c6e9462bd3da4a: Status 404 returned error can't find the container with id 57caebf9d4be4e71da611bcdb4ff1e9e98a44a56df4b95a556c6e9462bd3da4a
Sep 06 05:10:46.636186 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:46.636155    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:10:46.637925 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:46.637877    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3d364b_a587_4a9e_a247_c65dd68564b9.slice/crio-c2d651a6b8b156ad1fb9756dddac6a024b97819a5ba10f1500938bc9c428afb5 WatchSource:0}: Error finding container c2d651a6b8b156ad1fb9756dddac6a024b97819a5ba10f1500938bc9c428afb5: Status 404 returned error can't find the container with id c2d651a6b8b156ad1fb9756dddac6a024b97819a5ba10f1500938bc9c428afb5
Sep 06 05:10:47.051137 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.051094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" event={"ID":"19a82f33-a549-46e8-8272-6b82f4ef1b82","Type":"ContainerStarted","Data":"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"}
Sep 06 05:10:47.051603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.051155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" event={"ID":"19a82f33-a549-46e8-8272-6b82f4ef1b82","Type":"ContainerStarted","Data":"57caebf9d4be4e71da611bcdb4ff1e9e98a44a56df4b95a556c6e9462bd3da4a"}
Sep 06 05:10:47.051603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.051221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:47.052941 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.052913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" event={"ID":"2a3d364b-a587-4a9e-a247-c65dd68564b9","Type":"ContainerStarted","Data":"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"}
Sep 06 05:10:47.053087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.052958    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" event={"ID":"2a3d364b-a587-4a9e-a247-c65dd68564b9","Type":"ContainerStarted","Data":"c2d651a6b8b156ad1fb9756dddac6a024b97819a5ba10f1500938bc9c428afb5"}
Sep 06 05:10:47.053087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.053041    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:10:47.069627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.069546    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" podStartSLOduration=2.069522751 podStartE2EDuration="2.069522751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:47.068618249 +0000 UTC m=+909.157695979" watchObservedRunningTime="2026-09-06 05:10:47.069522751 +0000 UTC m=+909.158600447"
Sep 06 05:10:47.085237 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:47.085068    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" podStartSLOduration=1.085048461 podStartE2EDuration="1.085048461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:47.084168988 +0000 UTC m=+909.173246683" watchObservedRunningTime="2026-09-06 05:10:47.085048461 +0000 UTC m=+909.174126155"
Sep 06 05:10:48.368447 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.368415    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:10:48.372254 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.372223    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.375041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.375019    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-64514da2-openshift-default-dockercfg-mjx2s\""
Sep 06 05:10:48.387377 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.387348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:10:48.398761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.398733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.398941 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.398779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.398941 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.398808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5z85\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.398941 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.398882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.399064 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.398985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.399064 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.399031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.399164 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.399118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.399164 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.399158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.399262 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.399219    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5z85\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500207    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500268    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500369    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.500998 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.500972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.501268 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.501214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.501389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.501363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.501707 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.501688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.501956 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.501937    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.503796 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.503772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.504293 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.504273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.513493 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.513472    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5z85\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.513923 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.513878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token\") pod \"e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") " pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.685745 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.685710    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:48.958687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.958660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:10:48.960639 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:48.960611    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d33a2e_5de5_494c_b70a_f319a663f184.slice/crio-b80ced59bde042549a6f4759fc4981d7a28143dd1865206106b926e5f3dc7576 WatchSource:0}: Error finding container b80ced59bde042549a6f4759fc4981d7a28143dd1865206106b926e5f3dc7576: Status 404 returned error can't find the container with id b80ced59bde042549a6f4759fc4981d7a28143dd1865206106b926e5f3dc7576
Sep 06 05:10:48.962964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.962919    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:10:48.963072 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.962996    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:10:48.963072 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:48.963027    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:10:49.064336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:49.064298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerStarted","Data":"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"}
Sep 06 05:10:49.064527 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:49.064357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerStarted","Data":"b80ced59bde042549a6f4759fc4981d7a28143dd1865206106b926e5f3dc7576"}
Sep 06 05:10:49.081323 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:49.081222    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podStartSLOduration=1.081201669 podStartE2EDuration="1.081201669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:49.080831996 +0000 UTC m=+911.169909691" watchObservedRunningTime="2026-09-06 05:10:49.081201669 +0000 UTC m=+911.170279362"
Sep 06 05:10:49.686327 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:49.686301    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:50.690444 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:50.690413    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:51.074440 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:51.074359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:51.075567 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:51.075546    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:10:52.438419 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.438384    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:10:52.441725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.441705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.444318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.444290    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-64514da2\""
Sep 06 05:10:52.444318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.444304    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-64514da2-dockercfg-6llsn\""
Sep 06 05:10:52.449363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.449333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:10:52.538382 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.538349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxtvd\" (UniqueName: \"kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.538555 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.538528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.639893 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.639859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.640087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.639946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxtvd\" (UniqueName: \"kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.640601 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.640580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.648325 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.648301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxtvd\" (UniqueName: \"kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd\") pod \"payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:52.755475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:52.755375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:53.021996 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:53.021917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:10:53.025942 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:53.025895    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178f7721_9878_46e5_aa55_964437410ca5.slice/crio-7123f8e6fbebbe37bc507ac0a95e9db97057ceb01efdc60ea047b3e4da51b0f9 WatchSource:0}: Error finding container 7123f8e6fbebbe37bc507ac0a95e9db97057ceb01efdc60ea047b3e4da51b0f9: Status 404 returned error can't find the container with id 7123f8e6fbebbe37bc507ac0a95e9db97057ceb01efdc60ea047b3e4da51b0f9
Sep 06 05:10:53.085513 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:53.085482    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" event={"ID":"178f7721-9878-46e5-aa55-964437410ca5","Type":"ContainerStarted","Data":"7123f8e6fbebbe37bc507ac0a95e9db97057ceb01efdc60ea047b3e4da51b0f9"}
Sep 06 05:10:54.092152 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:54.092115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" event={"ID":"178f7721-9878-46e5-aa55-964437410ca5","Type":"ContainerStarted","Data":"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"}
Sep 06 05:10:54.092558 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:54.092186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:10:54.107961 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:54.107889    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" podStartSLOduration=2.107875285 podStartE2EDuration="2.107875285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:54.106952903 +0000 UTC m=+916.196030597" watchObservedRunningTime="2026-09-06 05:10:54.107875285 +0000 UTC m=+916.196952979"
Sep 06 05:10:55.190053 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.190019    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:10:55.193969 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.193952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.196769 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.196745    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-9afd7f\""
Sep 06 05:10:55.202355 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.202332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:10:55.220791 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.220761    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:10:55.224104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.224086    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.226492 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.226467    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-9afd7f-dockercfg-fkd8f\""
Sep 06 05:10:55.234433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.234415    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:10:55.267059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.267027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvmmz\" (UniqueName: \"kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.267059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.267074    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnfz2\" (UniqueName: \"kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.267298 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.267106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.267298 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.267125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.368135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.368135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.368399 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvmmz\" (UniqueName: \"kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.368399 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnfz2\" (UniqueName: \"kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.368812 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.368975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.368956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.375476 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.375455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvmmz\" (UniqueName: \"kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz\") pod \"payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") " pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.375559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.375513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnfz2\" (UniqueName: \"kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2\") pod \"payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.506415 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.506330    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:55.534752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.534719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:55.779550 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.779528    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:10:55.781700 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:55.781678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807ac03e_8c4e_4e93_8560_d34d05a734ad.slice/crio-33380819b19a39b2af6726005ebc08016ac028f2ba80a9e11d48c791373adf1d WatchSource:0}: Error finding container 33380819b19a39b2af6726005ebc08016ac028f2ba80a9e11d48c791373adf1d: Status 404 returned error can't find the container with id 33380819b19a39b2af6726005ebc08016ac028f2ba80a9e11d48c791373adf1d
Sep 06 05:10:55.830493 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:55.830461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:10:55.831673 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:10:55.831650    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b36caf_ceaa_4ff3_85d1_1342363a7bfa.slice/crio-9090a904203c98b17b60638f19ebc10d97d1954ced4bb4cbd1f94b26c1a9e175 WatchSource:0}: Error finding container 9090a904203c98b17b60638f19ebc10d97d1954ced4bb4cbd1f94b26c1a9e175: Status 404 returned error can't find the container with id 9090a904203c98b17b60638f19ebc10d97d1954ced4bb4cbd1f94b26c1a9e175
Sep 06 05:10:56.108181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.108075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" event={"ID":"84b36caf-ceaa-4ff3-85d1-1342363a7bfa","Type":"ContainerStarted","Data":"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"}
Sep 06 05:10:56.108181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.108140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" event={"ID":"84b36caf-ceaa-4ff3-85d1-1342363a7bfa","Type":"ContainerStarted","Data":"9090a904203c98b17b60638f19ebc10d97d1954ced4bb4cbd1f94b26c1a9e175"}
Sep 06 05:10:56.108181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.108172    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:10:56.110126 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.110100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" event={"ID":"807ac03e-8c4e-4e93-8560-d34d05a734ad","Type":"ContainerStarted","Data":"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"}
Sep 06 05:10:56.110273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.110148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" event={"ID":"807ac03e-8c4e-4e93-8560-d34d05a734ad","Type":"ContainerStarted","Data":"33380819b19a39b2af6726005ebc08016ac028f2ba80a9e11d48c791373adf1d"}
Sep 06 05:10:56.110273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.110264    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:10:56.125027 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.124987    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" podStartSLOduration=1.124971068 podStartE2EDuration="1.124971068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:56.124562142 +0000 UTC m=+918.213639838" watchObservedRunningTime="2026-09-06 05:10:56.124971068 +0000 UTC m=+918.214048761"
Sep 06 05:10:56.140483 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:56.140444    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" podStartSLOduration=1.140430138 podStartE2EDuration="1.140430138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:10:56.13823328 +0000 UTC m=+918.227310974" watchObservedRunningTime="2026-09-06 05:10:56.140430138 +0000 UTC m=+918.229507859"
Sep 06 05:10:58.059074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:58.059045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:10:58.059498 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:10:58.059348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:11:05.100334 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:05.100298    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:11:07.118359 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:07.118326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:11:07.118753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:07.118398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:11:14.202561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.202530    2576 generic.go:356] "Generic (PLEG): container finished" podID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerID="f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31" exitCode=0
Sep 06 05:11:14.203024 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.202599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerDied","Data":"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"}
Sep 06 05:11:14.203105 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.203091    2576 scope.go:117] "RemoveContainer" containerID="f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"
Sep 06 05:11:14.204187 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.204160    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:11:14.204260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.204234    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:11:14.204297 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.204269    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:11:14.541097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.541017    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:11:14.543485 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.543469    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.546024 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.545991    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-9035bc\""
Sep 06 05:11:14.546024 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.546005    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-9035bc-dockercfg-9t495\""
Sep 06 05:11:14.554078 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.554055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:11:14.557676 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.557234    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:11:14.562484 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.562450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.562648 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.562631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlp4r\" (UniqueName: \"kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.562820 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.562804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.568653 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.568617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:11:14.663852 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.663814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlp4r\" (UniqueName: \"kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.664073 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.663881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.664073 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.663931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.664073 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.663960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkpg9\" (UniqueName: \"kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.664632 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.664608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.673091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.673050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlp4r\" (UniqueName: \"kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r\") pod \"payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.764679 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.764647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.764884 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.764684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkpg9\" (UniqueName: \"kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.765400 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.765380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.775940 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.774700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkpg9\" (UniqueName: \"kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9\") pod \"payload-processing-e2e-inf-b-9035bc-645695948c-stzr7\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") " pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:14.858036 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.857936    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:14.881316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:14.881275    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:15.143489 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.143438    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:11:15.144888 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:11:15.144850    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6784308e_74bb_4e83_b284_9b9c01d88382.slice/crio-8acc67673e811ba670e5fc16837d983d0992c62e5ddfbf4a3f7660e8c9cb26f1 WatchSource:0}: Error finding container 8acc67673e811ba670e5fc16837d983d0992c62e5ddfbf4a3f7660e8c9cb26f1: Status 404 returned error can't find the container with id 8acc67673e811ba670e5fc16837d983d0992c62e5ddfbf4a3f7660e8c9cb26f1
Sep 06 05:11:15.168270 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.168244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:11:15.179128 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:11:15.179098    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78d79b3_d067_47aa_a856_51559edfae69.slice/crio-c0357611af9b874dcd29a1ce8af8b355e8fae969a83701df717df78236ccc111 WatchSource:0}: Error finding container c0357611af9b874dcd29a1ce8af8b355e8fae969a83701df717df78236ccc111: Status 404 returned error can't find the container with id c0357611af9b874dcd29a1ce8af8b355e8fae969a83701df717df78236ccc111
Sep 06 05:11:15.210859 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.210796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerStarted","Data":"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"}
Sep 06 05:11:15.212948 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.212888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" event={"ID":"b78d79b3-d067-47aa-a856-51559edfae69","Type":"ContainerStarted","Data":"c0357611af9b874dcd29a1ce8af8b355e8fae969a83701df717df78236ccc111"}
Sep 06 05:11:15.214517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.214491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" event={"ID":"6784308e-74bb-4e83-b284-9b9c01d88382","Type":"ContainerStarted","Data":"8acc67673e811ba670e5fc16837d983d0992c62e5ddfbf4a3f7660e8c9cb26f1"}
Sep 06 05:11:15.686461 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.686428    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:15.686818 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.686792    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:15.686891 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.686855    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:15.896953 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.896893    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"]
Sep 06 05:11:15.900336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.900306    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:15.902725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.902701    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:11:15.907525 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.907501    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"]
Sep 06 05:11:15.974413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:15.974303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-64514da2-n7pbl\" (UID: \"f923168f-1d7a-4729-9dec-2c08db9d5b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:16.075792 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.075745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-64514da2-n7pbl\" (UID: \"f923168f-1d7a-4729-9dec-2c08db9d5b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:16.076736 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.076703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-64514da2-n7pbl\" (UID: \"f923168f-1d7a-4729-9dec-2c08db9d5b39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:16.214104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.214058    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:16.220793 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.220760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" event={"ID":"b78d79b3-d067-47aa-a856-51559edfae69","Type":"ContainerStarted","Data":"2b751aba1b49cf470fb73ff22d3653f6b8720f31fd789cc9fe5fd3116e44e0f4"}
Sep 06 05:11:16.220959 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.220938    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:16.222540 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.222513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" event={"ID":"6784308e-74bb-4e83-b284-9b9c01d88382","Type":"ContainerStarted","Data":"eea215d92d8f245ab8742f63a0701ce76f83a5245a2a4a6cb62a77d514bdf5c0"}
Sep 06 05:11:16.222650 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.222603    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:16.244121 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.244061    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" podStartSLOduration=2.244039994 podStartE2EDuration="2.244039994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:16.235308575 +0000 UTC m=+938.324386291" watchObservedRunningTime="2026-09-06 05:11:16.244039994 +0000 UTC m=+938.333117687"
Sep 06 05:11:16.255299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.255232    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" podStartSLOduration=2.255211502 podStartE2EDuration="2.255211502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:16.253626881 +0000 UTC m=+938.342704594" watchObservedRunningTime="2026-09-06 05:11:16.255211502 +0000 UTC m=+938.344289196"
Sep 06 05:11:16.498700 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.498673    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"]
Sep 06 05:11:16.500187 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:11:16.500154    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf923168f_1d7a_4729_9dec_2c08db9d5b39.slice/crio-1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6 WatchSource:0}: Error finding container 1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6: Status 404 returned error can't find the container with id 1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6
Sep 06 05:11:16.686765 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.686290    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:16.686765 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:16.686362    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:17.229003 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:17.228966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl" event={"ID":"f923168f-1d7a-4729-9dec-2c08db9d5b39","Type":"ContainerStarted","Data":"1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6"}
Sep 06 05:11:17.686377 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:17.686340    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:17.686640 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:17.686613    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:18.693412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:18.693371    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:18.693874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:18.693442    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:19.687536 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:19.687063    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:19.687536 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:19.687123    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:20.247591 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:20.247560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl" event={"ID":"f923168f-1d7a-4729-9dec-2c08db9d5b39","Type":"ContainerStarted","Data":"ec5fd66f05abf4011681b9d6e0f4ec686161b20eec16d2857ccf8f75f19fab5f"}
Sep 06 05:11:20.267362 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:20.267284    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl" podStartSLOduration=2.253372568 podStartE2EDuration="5.267262137s" totalImagesPullingTime="3.013889569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:11:20.265670456 +0000 UTC m=+942.354748163" watchObservedRunningTime="2026-09-06 05:11:20.267262137 +0000 UTC m=+942.356339831"
Sep 06 05:11:20.686615 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:20.686567    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 06 05:11:20.686826 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:20.686641    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 06 05:11:21.075113 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:21.075072    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:21.253552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:21.253513    2576 generic.go:356] "Generic (PLEG): container finished" podID="f923168f-1d7a-4729-9dec-2c08db9d5b39" containerID="ec5fd66f05abf4011681b9d6e0f4ec686161b20eec16d2857ccf8f75f19fab5f" exitCode=0
Sep 06 05:11:21.253986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:21.253606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl" event={"ID":"f923168f-1d7a-4729-9dec-2c08db9d5b39","Type":"ContainerDied","Data":"ec5fd66f05abf4011681b9d6e0f4ec686161b20eec16d2857ccf8f75f19fab5f"}
Sep 06 05:11:21.690544 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:21.690516    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:21.691669 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:21.691643    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:22.400665 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:22.400633    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:22.451701 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:22.451667    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca\") pod \"f923168f-1d7a-4729-9dec-2c08db9d5b39\" (UID: \"f923168f-1d7a-4729-9dec-2c08db9d5b39\") "
Sep 06 05:11:22.452009 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:22.451986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f923168f-1d7a-4729-9dec-2c08db9d5b39" (UID: "f923168f-1d7a-4729-9dec-2c08db9d5b39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:11:22.552592 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:22.552504    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f923168f-1d7a-4729-9dec-2c08db9d5b39-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:23.265196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:23.265170    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"
Sep 06 05:11:23.265382 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:23.265215    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl" event={"ID":"f923168f-1d7a-4729-9dec-2c08db9d5b39","Type":"ContainerDied","Data":"1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6"}
Sep 06 05:11:23.265382 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:23.265258    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d56f04ff852aa8ff668303b2772e450de94c8d8911d24d219cd5bd6851662f6"
Sep 06 05:11:27.231420 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:27.231391    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:11:27.231811 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:27.231457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:11:32.445736 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.445696    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:11:32.446290 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.445991    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" podUID="178f7721-9878-46e5-aa55-964437410ca5" containerName="payload-pre-processing" containerID="cri-o://e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8" gracePeriod=30
Sep 06 05:11:32.594789 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.594720    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"]
Sep 06 05:11:32.596805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.596762    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-64514da2-n7pbl"]
Sep 06 05:11:32.711634 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.711587    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:11:32.712641 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.711859    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" containerID="cri-o://47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261" gracePeriod=30
Sep 06 05:11:32.750985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.750946    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:11:32.855506 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.855477    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume\") pod \"178f7721-9878-46e5-aa55-964437410ca5\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") "
Sep 06 05:11:32.855705 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.855585    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxtvd\" (UniqueName: \"kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd\") pod \"178f7721-9878-46e5-aa55-964437410ca5\" (UID: \"178f7721-9878-46e5-aa55-964437410ca5\") "
Sep 06 05:11:32.855939 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.855882    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "178f7721-9878-46e5-aa55-964437410ca5" (UID: "178f7721-9878-46e5-aa55-964437410ca5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:11:32.858051 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.858027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd" (OuterVolumeSpecName: "kube-api-access-qxtvd") pod "178f7721-9878-46e5-aa55-964437410ca5" (UID: "178f7721-9878-46e5-aa55-964437410ca5"). InnerVolumeSpecName "kube-api-access-qxtvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:11:32.956695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.956597    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/178f7721-9878-46e5-aa55-964437410ca5-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:32.956695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:32.956642    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxtvd\" (UniqueName: \"kubernetes.io/projected/178f7721-9878-46e5-aa55-964437410ca5-kube-api-access-qxtvd\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:33.322595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.322513    2576 generic.go:356] "Generic (PLEG): container finished" podID="178f7721-9878-46e5-aa55-964437410ca5" containerID="e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8" exitCode=0
Sep 06 05:11:33.322595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.322564    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"
Sep 06 05:11:33.322778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.322597    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" event={"ID":"178f7721-9878-46e5-aa55-964437410ca5","Type":"ContainerDied","Data":"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"}
Sep 06 05:11:33.322778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.322653    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k" event={"ID":"178f7721-9878-46e5-aa55-964437410ca5","Type":"ContainerDied","Data":"7123f8e6fbebbe37bc507ac0a95e9db97057ceb01efdc60ea047b3e4da51b0f9"}
Sep 06 05:11:33.322778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.322677    2576 scope.go:117] "RemoveContainer" containerID="e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"
Sep 06 05:11:33.331944 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.331925    2576 scope.go:117] "RemoveContainer" containerID="e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"
Sep 06 05:11:33.332189 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:11:33.332171    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8\": container with ID starting with e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8 not found: ID does not exist" containerID="e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"
Sep 06 05:11:33.332263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.332197    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8"} err="failed to get container status \"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8\": rpc error: code = NotFound desc = could not find container \"e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8\": container with ID starting with e3c1721cc6807492360720f83c50cde7f76900eeac4f9e06a64fb1e9082d7fd8 not found: ID does not exist"
Sep 06 05:11:33.344396 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.344376    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:11:33.348326 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:33.348305    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-64514da2-59b8cbf77d-4rq7k"]
Sep 06 05:11:34.387188 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:34.387155    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="178f7721-9878-46e5-aa55-964437410ca5" path="/var/lib/kubelet/pods/178f7721-9878-46e5-aa55-964437410ca5/volumes"
Sep 06 05:11:34.387562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:34.387489    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f923168f-1d7a-4729-9dec-2c08db9d5b39" path="/var/lib/kubelet/pods/f923168f-1d7a-4729-9dec-2c08db9d5b39/volumes"
Sep 06 05:11:36.692972 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:36.692938    2576 patch_prober.go:28] interesting pod/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:11:36.693334 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:36.693002    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:11:37.975025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:37.974968    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:38.112502 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112463    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5z85\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112562    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112577    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112620    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112745    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket\") pod \"b5d33a2e-5de5-494c-b70a-f319a663f184\" (UID: \"b5d33a2e-5de5-494c-b70a-f319a663f184\") "
Sep 06 05:11:38.112985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.112969    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:11:38.113204 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.113093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:11:38.113267 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.113205    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:11:38.113267 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.113234    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:11:38.113267 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.113208    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.113583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.113357    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data" (OuterVolumeSpecName: "istio-data") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:11:38.114975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.114947    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:11:38.115087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.115034    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:11:38.115087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.115034    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token" (OuterVolumeSpecName: "istio-token") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:11:38.115087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.115065    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85" (OuterVolumeSpecName: "kube-api-access-b5z85") pod "b5d33a2e-5de5-494c-b70a-f319a663f184" (UID: "b5d33a2e-5de5-494c-b70a-f319a663f184"). InnerVolumeSpecName "kube-api-access-b5z85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:11:38.214182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214149    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5z85\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-kube-api-access-b5z85\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214177    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214187    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214196    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b5d33a2e-5de5-494c-b70a-f319a663f184-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214205    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214213    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214221    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.214398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.214230    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b5d33a2e-5de5-494c-b70a-f319a663f184-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:11:38.347794 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.347721    2576 generic.go:356] "Generic (PLEG): container finished" podID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerID="47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261" exitCode=0
Sep 06 05:11:38.347794 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.347774    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"
Sep 06 05:11:38.347988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.347802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerDied","Data":"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"}
Sep 06 05:11:38.347988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.347889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4" event={"ID":"b5d33a2e-5de5-494c-b70a-f319a663f184","Type":"ContainerDied","Data":"b80ced59bde042549a6f4759fc4981d7a28143dd1865206106b926e5f3dc7576"}
Sep 06 05:11:38.347988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.347940    2576 scope.go:117] "RemoveContainer" containerID="47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"
Sep 06 05:11:38.357470 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.357449    2576 scope.go:117] "RemoveContainer" containerID="f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"
Sep 06 05:11:38.365513 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.365497    2576 scope.go:117] "RemoveContainer" containerID="47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"
Sep 06 05:11:38.365742 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:11:38.365725    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261\": container with ID starting with 47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261 not found: ID does not exist" containerID="47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"
Sep 06 05:11:38.365797 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.365750    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261"} err="failed to get container status \"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261\": rpc error: code = NotFound desc = could not find container \"47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261\": container with ID starting with 47edc20ef4b9111131ac7fd560103305a801e420a92df040790765c5fb47f261 not found: ID does not exist"
Sep 06 05:11:38.365797 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.365766    2576 scope.go:117] "RemoveContainer" containerID="f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"
Sep 06 05:11:38.366081 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:11:38.366054    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31\": container with ID starting with f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31 not found: ID does not exist" containerID="f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"
Sep 06 05:11:38.366228 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.366091    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31"} err="failed to get container status \"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31\": rpc error: code = NotFound desc = could not find container \"f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31\": container with ID starting with f576d6c779505cf059a9c206c05f579bbf09014048dd1cc7d84d74e81c9fbc31 not found: ID does not exist"
Sep 06 05:11:38.368758 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.368738    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:11:38.375393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.375371    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-64514da2-openshift-default-594dfccc66-wv7x4"]
Sep 06 05:11:38.385144 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:11:38.385125    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" path="/var/lib/kubelet/pods/b5d33a2e-5de5-494c-b70a-f319a663f184/volumes"
Sep 06 05:12:01.047473 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047394    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"]
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047767    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047778    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047794    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047800    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047810    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="178f7721-9878-46e5-aa55-964437410ca5" containerName="payload-pre-processing"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047817    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="178f7721-9878-46e5-aa55-964437410ca5" containerName="payload-pre-processing"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047826    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f923168f-1d7a-4729-9dec-2c08db9d5b39" containerName="revoke-keys"
Sep 06 05:12:01.047869 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047832    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f923168f-1d7a-4729-9dec-2c08db9d5b39" containerName="revoke-keys"
Sep 06 05:12:01.048139 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047885    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f923168f-1d7a-4729-9dec-2c08db9d5b39" containerName="revoke-keys"
Sep 06 05:12:01.048139 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047894    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.048139 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047915    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="178f7721-9878-46e5-aa55-964437410ca5" containerName="payload-pre-processing"
Sep 06 05:12:01.048139 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.047924    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d33a2e-5de5-494c-b70a-f319a663f184" containerName="istio-proxy"
Sep 06 05:12:01.051067 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.051050    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:01.053433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.053416    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:12:01.057374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.057348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"]
Sep 06 05:12:01.124813 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.124783    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-85214949-nz4nh\" (UID: \"884ffca8-37b5-4237-8a3c-e1f0bb83dd02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:01.225298 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.225260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-85214949-nz4nh\" (UID: \"884ffca8-37b5-4237-8a3c-e1f0bb83dd02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:01.225886 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.225865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-85214949-nz4nh\" (UID: \"884ffca8-37b5-4237-8a3c-e1f0bb83dd02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:01.362836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.362757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:01.629545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:01.629485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"]
Sep 06 05:12:01.632020 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:01.631989    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884ffca8_37b5_4237_8a3c_e1f0bb83dd02.slice/crio-2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a WatchSource:0}: Error finding container 2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a: Status 404 returned error can't find the container with id 2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a
Sep 06 05:12:02.464539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:02.464505    2576 generic.go:356] "Generic (PLEG): container finished" podID="884ffca8-37b5-4237-8a3c-e1f0bb83dd02" containerID="3b5bfff6f50bde1adf9854de1d46a4077c11f3a17db4966b467a3257e62e5dd4" exitCode=0
Sep 06 05:12:02.464935 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:02.464598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh" event={"ID":"884ffca8-37b5-4237-8a3c-e1f0bb83dd02","Type":"ContainerDied","Data":"3b5bfff6f50bde1adf9854de1d46a4077c11f3a17db4966b467a3257e62e5dd4"}
Sep 06 05:12:02.464935 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:02.464649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh" event={"ID":"884ffca8-37b5-4237-8a3c-e1f0bb83dd02","Type":"ContainerStarted","Data":"2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a"}
Sep 06 05:12:03.607444 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:03.607420    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:03.647454 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:03.647425    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca\") pod \"884ffca8-37b5-4237-8a3c-e1f0bb83dd02\" (UID: \"884ffca8-37b5-4237-8a3c-e1f0bb83dd02\") "
Sep 06 05:12:03.647777 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:03.647754    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "884ffca8-37b5-4237-8a3c-e1f0bb83dd02" (UID: "884ffca8-37b5-4237-8a3c-e1f0bb83dd02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:03.749048 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:03.748965    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/884ffca8-37b5-4237-8a3c-e1f0bb83dd02-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:04.475530 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:04.475502    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"
Sep 06 05:12:04.475530 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:04.475505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh" event={"ID":"884ffca8-37b5-4237-8a3c-e1f0bb83dd02","Type":"ContainerDied","Data":"2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a"}
Sep 06 05:12:04.475742 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:04.475552    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fe3ad95e6849c93ca27e760656e1c6c1d9b365bb3117bfce03b46684b62be3a"
Sep 06 05:12:07.841919 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:07.841869    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"]
Sep 06 05:12:07.843595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:07.843569    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-85214949-nz4nh"]
Sep 06 05:12:08.387215 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:08.387181    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="884ffca8-37b5-4237-8a3c-e1f0bb83dd02" path="/var/lib/kubelet/pods/884ffca8-37b5-4237-8a3c-e1f0bb83dd02/volumes"
Sep 06 05:12:13.073781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.073744    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:13.074246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.074197    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="884ffca8-37b5-4237-8a3c-e1f0bb83dd02" containerName="revoke-keys"
Sep 06 05:12:13.074246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.074210    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="884ffca8-37b5-4237-8a3c-e1f0bb83dd02" containerName="revoke-keys"
Sep 06 05:12:13.074332 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.074288    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="884ffca8-37b5-4237-8a3c-e1f0bb83dd02" containerName="revoke-keys"
Sep 06 05:12:13.080067 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.079073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.083134 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.083110    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a4f2603a-dockercfg-fb9vb\""
Sep 06 05:12:13.083291 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.083153    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a4f2603a\""
Sep 06 05:12:13.087606 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.087581    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:13.106809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.106772    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:13.111128 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.111102    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.119467 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.119440    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:13.133354 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.133319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.133526 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.133374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6sbl\" (UniqueName: \"kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.234547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.234514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l727d\" (UniqueName: \"kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.234809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.234577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.234809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.234649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6sbl\" (UniqueName: \"kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.234809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.234712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.235356 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.235338    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.242776 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.242746    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6sbl\" (UniqueName: \"kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl\") pod \"payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.336400 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.336297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.336400 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.336378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l727d\" (UniqueName: \"kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.337035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.337016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.343733 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.343707    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l727d\" (UniqueName: \"kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d\") pod \"payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") " pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.393476 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.393444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:13.422850 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.422817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:13.666705 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.666682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:13.669097 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:13.669072    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7fff0b_7b3c_4c2a_aaeb_1addace7c923.slice/crio-7c4b6a26c2781ded93d9e98700e98f7ffe0ab7a7f69e058c473fd836985b00f5 WatchSource:0}: Error finding container 7c4b6a26c2781ded93d9e98700e98f7ffe0ab7a7f69e058c473fd836985b00f5: Status 404 returned error can't find the container with id 7c4b6a26c2781ded93d9e98700e98f7ffe0ab7a7f69e058c473fd836985b00f5
Sep 06 05:12:13.702519 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:13.702496    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:13.704326 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:13.704301    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebaeac2_744e_4eb8_812d_b27b965e864a.slice/crio-e1a43766f7ba424305ba1e84af364298f4fa95000ac03dc8a06ff7bc47818ea4 WatchSource:0}: Error finding container e1a43766f7ba424305ba1e84af364298f4fa95000ac03dc8a06ff7bc47818ea4: Status 404 returned error can't find the container with id e1a43766f7ba424305ba1e84af364298f4fa95000ac03dc8a06ff7bc47818ea4
Sep 06 05:12:14.526675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.526638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" event={"ID":"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923","Type":"ContainerStarted","Data":"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"}
Sep 06 05:12:14.527169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.526703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" event={"ID":"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923","Type":"ContainerStarted","Data":"7c4b6a26c2781ded93d9e98700e98f7ffe0ab7a7f69e058c473fd836985b00f5"}
Sep 06 05:12:14.527169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.526835    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:14.528404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.528378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" event={"ID":"2ebaeac2-744e-4eb8-812d-b27b965e864a","Type":"ContainerStarted","Data":"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"}
Sep 06 05:12:14.528550 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.528424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" event={"ID":"2ebaeac2-744e-4eb8-812d-b27b965e864a","Type":"ContainerStarted","Data":"e1a43766f7ba424305ba1e84af364298f4fa95000ac03dc8a06ff7bc47818ea4"}
Sep 06 05:12:14.528550 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.528498    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:14.542484 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.542426    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" podStartSLOduration=1.5424070300000001 podStartE2EDuration="1.54240703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:14.541306072 +0000 UTC m=+996.630383766" watchObservedRunningTime="2026-09-06 05:12:14.54240703 +0000 UTC m=+996.631484726"
Sep 06 05:12:14.557387 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:14.557320    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" podStartSLOduration=1.557301439 podStartE2EDuration="1.557301439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:14.555302708 +0000 UTC m=+996.644380413" watchObservedRunningTime="2026-09-06 05:12:14.557301439 +0000 UTC m=+996.646379132"
Sep 06 05:12:25.535114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:25.535083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:25.535621 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:25.535401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:37.003266 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.003226    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"]
Sep 06 05:12:37.006889 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.006872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:37.009384 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.009365    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:12:37.013333 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.013307    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"]
Sep 06 05:12:37.073709 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.073653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp\" (UID: \"c4d21bae-9da0-4e7e-a36e-6905e6e0639d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:37.174164 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.174130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp\" (UID: \"c4d21bae-9da0-4e7e-a36e-6905e6e0639d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:37.174803 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.174779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp\" (UID: \"c4d21bae-9da0-4e7e-a36e-6905e6e0639d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:37.319677 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.319596    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:37.593299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.593266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"]
Sep 06 05:12:37.594827 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:37.594783    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d21bae_9da0_4e7e_a36e_6905e6e0639d.slice/crio-4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f WatchSource:0}: Error finding container 4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f: Status 404 returned error can't find the container with id 4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f
Sep 06 05:12:37.643678 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:37.643646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp" event={"ID":"c4d21bae-9da0-4e7e-a36e-6905e6e0639d","Type":"ContainerStarted","Data":"4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f"}
Sep 06 05:12:38.650142 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:38.650046    2576 generic.go:356] "Generic (PLEG): container finished" podID="c4d21bae-9da0-4e7e-a36e-6905e6e0639d" containerID="c5c6e55acbd23e83b1a0ec604039631749dbd9467935a2b0b6855a7d49449f85" exitCode=0
Sep 06 05:12:38.650142 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:38.650112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp" event={"ID":"c4d21bae-9da0-4e7e-a36e-6905e6e0639d","Type":"ContainerDied","Data":"c5c6e55acbd23e83b1a0ec604039631749dbd9467935a2b0b6855a7d49449f85"}
Sep 06 05:12:39.796437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:39.796414    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:39.900808 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:39.900778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca\") pod \"c4d21bae-9da0-4e7e-a36e-6905e6e0639d\" (UID: \"c4d21bae-9da0-4e7e-a36e-6905e6e0639d\") "
Sep 06 05:12:39.901177 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:39.901151    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c4d21bae-9da0-4e7e-a36e-6905e6e0639d" (UID: "c4d21bae-9da0-4e7e-a36e-6905e6e0639d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:40.002336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:40.002293    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4d21bae-9da0-4e7e-a36e-6905e6e0639d-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:40.662311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:40.662218    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp" event={"ID":"c4d21bae-9da0-4e7e-a36e-6905e6e0639d","Type":"ContainerDied","Data":"4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f"}
Sep 06 05:12:40.662311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:40.662259    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"
Sep 06 05:12:40.662311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:40.662275    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dc9b1e1e776fa53f9c1c75588be3b2ba0b071b5b64eefe441052692e8c1b38f"
Sep 06 05:12:43.027524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.027489    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:43.027945 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.027768    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" podUID="2ebaeac2-744e-4eb8-812d-b27b965e864a" containerName="payload-processing" containerID="cri-o://af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f" gracePeriod=30
Sep 06 05:12:43.032477 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.032451    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:43.032873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.032848    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" podUID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" containerName="payload-pre-processing" containerID="cri-o://02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5" gracePeriod=30
Sep 06 05:12:43.170873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.170827    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"]
Sep 06 05:12:43.172573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.172547    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a4f2603a-sdbsp"]
Sep 06 05:12:43.312962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.312938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:43.315652 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.315636    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:43.436611 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.436576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l727d\" (UniqueName: \"kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d\") pod \"2ebaeac2-744e-4eb8-812d-b27b965e864a\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") "
Sep 06 05:12:43.436818 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.436666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume\") pod \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") "
Sep 06 05:12:43.436818 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.436702    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6sbl\" (UniqueName: \"kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl\") pod \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\" (UID: \"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923\") "
Sep 06 05:12:43.436818 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.436728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume\") pod \"2ebaeac2-744e-4eb8-812d-b27b965e864a\" (UID: \"2ebaeac2-744e-4eb8-812d-b27b965e864a\") "
Sep 06 05:12:43.437186 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.437158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" (UID: "8f7fff0b-7b3c-4c2a-aaeb-1addace7c923"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:43.437919 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.437878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2ebaeac2-744e-4eb8-812d-b27b965e864a" (UID: "2ebaeac2-744e-4eb8-812d-b27b965e864a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:12:43.439577 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.439548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d" (OuterVolumeSpecName: "kube-api-access-l727d") pod "2ebaeac2-744e-4eb8-812d-b27b965e864a" (UID: "2ebaeac2-744e-4eb8-812d-b27b965e864a"). InnerVolumeSpecName "kube-api-access-l727d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:43.439684 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.439587    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl" (OuterVolumeSpecName: "kube-api-access-m6sbl") pod "8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" (UID: "8f7fff0b-7b3c-4c2a-aaeb-1addace7c923"). InnerVolumeSpecName "kube-api-access-m6sbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:12:43.537600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.537497    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:43.537600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.537538    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6sbl\" (UniqueName: \"kubernetes.io/projected/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923-kube-api-access-m6sbl\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:43.537600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.537548    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ebaeac2-744e-4eb8-812d-b27b965e864a-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:43.537600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.537556    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l727d\" (UniqueName: \"kubernetes.io/projected/2ebaeac2-744e-4eb8-812d-b27b965e864a-kube-api-access-l727d\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:12:43.677161 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.677124    2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaeac2-744e-4eb8-812d-b27b965e864a" containerID="af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f" exitCode=0
Sep 06 05:12:43.677361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.677181    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"
Sep 06 05:12:43.677361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.677211    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" event={"ID":"2ebaeac2-744e-4eb8-812d-b27b965e864a","Type":"ContainerDied","Data":"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"}
Sep 06 05:12:43.677361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.677272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9" event={"ID":"2ebaeac2-744e-4eb8-812d-b27b965e864a","Type":"ContainerDied","Data":"e1a43766f7ba424305ba1e84af364298f4fa95000ac03dc8a06ff7bc47818ea4"}
Sep 06 05:12:43.677361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.677305    2576 scope.go:117] "RemoveContainer" containerID="af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"
Sep 06 05:12:43.678663 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.678640    2576 generic.go:356] "Generic (PLEG): container finished" podID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" containerID="02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5" exitCode=0
Sep 06 05:12:43.678752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.678693    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"
Sep 06 05:12:43.678752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.678726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" event={"ID":"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923","Type":"ContainerDied","Data":"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"}
Sep 06 05:12:43.678849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.678779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2" event={"ID":"8f7fff0b-7b3c-4c2a-aaeb-1addace7c923","Type":"ContainerDied","Data":"7c4b6a26c2781ded93d9e98700e98f7ffe0ab7a7f69e058c473fd836985b00f5"}
Sep 06 05:12:43.686912 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.686879    2576 scope.go:117] "RemoveContainer" containerID="af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"
Sep 06 05:12:43.687187 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:12:43.687168    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f\": container with ID starting with af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f not found: ID does not exist" containerID="af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"
Sep 06 05:12:43.687236 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.687195    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f"} err="failed to get container status \"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f\": rpc error: code = NotFound desc = could not find container \"af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f\": container with ID starting with af34956f2c473782061aa93ee4f026edf72a8c5bf6518ab339ba407070a1473f not found: ID does not exist"
Sep 06 05:12:43.687236 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.687210    2576 scope.go:117] "RemoveContainer" containerID="02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"
Sep 06 05:12:43.695523 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.695505    2576 scope.go:117] "RemoveContainer" containerID="02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"
Sep 06 05:12:43.695806 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:12:43.695788    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5\": container with ID starting with 02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5 not found: ID does not exist" containerID="02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"
Sep 06 05:12:43.695874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.695816    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5"} err="failed to get container status \"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5\": rpc error: code = NotFound desc = could not find container \"02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5\": container with ID starting with 02cb3a830e43c5c1a7aea3aa53cee24be476598a447691b0447b0df32e0c7fa5 not found: ID does not exist"
Sep 06 05:12:43.700914 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.700879    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:43.704050 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.704028    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a4f2603a-855648954b-kk5c2"]
Sep 06 05:12:43.715763 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.715735    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:43.718973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:43.718951    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a4f2603a-5554c4458f-msgj9"]
Sep 06 05:12:44.389202 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.389101    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ebaeac2-744e-4eb8-812d-b27b965e864a" path="/var/lib/kubelet/pods/2ebaeac2-744e-4eb8-812d-b27b965e864a/volumes"
Sep 06 05:12:44.389836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.389810    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" path="/var/lib/kubelet/pods/8f7fff0b-7b3c-4c2a-aaeb-1addace7c923/volumes"
Sep 06 05:12:44.390498 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.390478    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4d21bae-9da0-4e7e-a36e-6905e6e0639d" path="/var/lib/kubelet/pods/c4d21bae-9da0-4e7e-a36e-6905e6e0639d/volumes"
Sep 06 05:12:44.935196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935163    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:12:44.935759 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935738    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" containerName="payload-pre-processing"
Sep 06 05:12:44.935831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935764    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" containerName="payload-pre-processing"
Sep 06 05:12:44.935831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935788    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaeac2-744e-4eb8-812d-b27b965e864a" containerName="payload-processing"
Sep 06 05:12:44.935831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935797    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaeac2-744e-4eb8-812d-b27b965e864a" containerName="payload-processing"
Sep 06 05:12:44.935831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935822    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4d21bae-9da0-4e7e-a36e-6905e6e0639d" containerName="revoke-keys"
Sep 06 05:12:44.935831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4d21bae-9da0-4e7e-a36e-6905e6e0639d" containerName="revoke-keys"
Sep 06 05:12:44.936115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935945    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f7fff0b-7b3c-4c2a-aaeb-1addace7c923" containerName="payload-pre-processing"
Sep 06 05:12:44.936115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935960    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4d21bae-9da0-4e7e-a36e-6905e6e0639d" containerName="revoke-keys"
Sep 06 05:12:44.936115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.935970    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebaeac2-744e-4eb8-812d-b27b965e864a" containerName="payload-processing"
Sep 06 05:12:44.941024 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.941000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:44.943475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.943456    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-de5ebc7b-openshift-default-dockercfg-lbkd8\""
Sep 06 05:12:44.949148 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:44.949124    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:12:45.053811 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.053778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.053824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.053856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.053883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.053936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c92v\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.054043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.054119    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.054160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.054288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.054230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.155583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.155778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.155778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.155778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c92v\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.155778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156013 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.155946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156070 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156070 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156387 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.156460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.156437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.158467 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.158446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.158578 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.158546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.163762 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.163732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.163891 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.163780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c92v\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v\") pod \"e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") " pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.252456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.252367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:45.528123 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.528091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:12:45.530257 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:45.530228    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4adf83e_ad8e_460f_92e5_7beb387b49d2.slice/crio-17a066915fa4a21c2feabb56a22a80fda600b65192bffba982f0e71bef09b35b WatchSource:0}: Error finding container 17a066915fa4a21c2feabb56a22a80fda600b65192bffba982f0e71bef09b35b: Status 404 returned error can't find the container with id 17a066915fa4a21c2feabb56a22a80fda600b65192bffba982f0e71bef09b35b
Sep 06 05:12:45.532544 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.532511    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:12:45.532611 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.532579    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:12:45.532658 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.532627    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:12:45.692951 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.692918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerStarted","Data":"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"}
Sep 06 05:12:45.693127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.692969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerStarted","Data":"17a066915fa4a21c2feabb56a22a80fda600b65192bffba982f0e71bef09b35b"}
Sep 06 05:12:45.713268 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:45.713201    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podStartSLOduration=1.713177049 podStartE2EDuration="1.713177049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:45.710670071 +0000 UTC m=+1027.799747786" watchObservedRunningTime="2026-09-06 05:12:45.713177049 +0000 UTC m=+1027.802254751"
Sep 06 05:12:46.252737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:46.252702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:46.258026 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:46.258000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:46.697978 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:46.697953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:46.699038 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:46.699016    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:12:48.790656 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.790626    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:12:48.794146 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.794130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:48.796389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.796372    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-de5ebc7b\""
Sep 06 05:12:48.796502 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.796377    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-de5ebc7b-dockercfg-kwx7j\""
Sep 06 05:12:48.804398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.804373    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:12:48.888573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.888542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:48.888770 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.888595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwmq5\" (UniqueName: \"kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:48.990110 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.990068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:48.990291 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.990132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwmq5\" (UniqueName: \"kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:48.990767 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:48.990749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:49.002968 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.002930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwmq5\" (UniqueName: \"kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5\") pod \"payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") " pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:49.106537 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.106441    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:12:49.379986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.379960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:12:49.382147 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:12:49.382117    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d27a57_4b57_43ca_bc60_748a7c69bc2b.slice/crio-26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1 WatchSource:0}: Error finding container 26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1: Status 404 returned error can't find the container with id 26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1
Sep 06 05:12:49.713683 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.713652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" event={"ID":"a8d27a57-4b57-43ca-bc60-748a7c69bc2b","Type":"ContainerStarted","Data":"cb6db4fa92d27fc72337e4bf3951b9a91ab97ee9aec130ed3c487c9200b86f4b"}
Sep 06 05:12:49.713920 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.713715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" event={"ID":"a8d27a57-4b57-43ca-bc60-748a7c69bc2b","Type":"ContainerStarted","Data":"26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1"}
Sep 06 05:12:49.713920 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:12:49.713746    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:13:00.720429 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:00.720400    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:13:00.736426 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:00.736371    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" podStartSLOduration=12.736355982 podStartE2EDuration="12.736355982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:12:49.728603735 +0000 UTC m=+1031.817681429" watchObservedRunningTime="2026-09-06 05:13:00.736355982 +0000 UTC m=+1042.825433676"
Sep 06 05:13:06.815224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.815190    2576 generic.go:356] "Generic (PLEG): container finished" podID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerID="9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331" exitCode=0
Sep 06 05:13:06.815681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.815260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerDied","Data":"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"}
Sep 06 05:13:06.815681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.815670    2576 scope.go:117] "RemoveContainer" containerID="9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"
Sep 06 05:13:06.816774 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.816746    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:13:06.816846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.816822    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:13:06.816942 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:06.816858    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:13:07.821768 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:07.821730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerStarted","Data":"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"}
Sep 06 05:13:08.253379 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:08.253335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:08.253642 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:08.253528    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:08.253642 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:08.253574    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:09.252929 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:09.252866    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:09.253399 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:09.252949    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:10.253693 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:10.253652    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:10.254223 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:10.253720    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:11.252815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.252775    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:11.253036 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.252834    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:11.888566 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.888498    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"]
Sep 06 05:13:11.891871 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.891844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:11.895137 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.895097    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:13:11.900018 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.899989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"]
Sep 06 05:13:11.920532 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:11.920500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q\" (UID: \"97459442-f194-4b5d-a09a-8f09632a4603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:12.022010 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.021976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q\" (UID: \"97459442-f194-4b5d-a09a-8f09632a4603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:12.022609 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.022588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q\" (UID: \"97459442-f194-4b5d-a09a-8f09632a4603\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:12.205352 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.205321    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:12.253866 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.253787    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:12.253866 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.253850    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:12.479980 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.479380    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"]
Sep 06 05:13:12.854981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.854944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q" event={"ID":"97459442-f194-4b5d-a09a-8f09632a4603","Type":"ContainerStarted","Data":"fd9c6e111f1d030f9603209d2ea33d067f708aea467f09eb75466c673cb4e4f4"}
Sep 06 05:13:12.855184 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:12.855008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q" event={"ID":"97459442-f194-4b5d-a09a-8f09632a4603","Type":"ContainerStarted","Data":"2c31f801362519586ab4f3fee76a2b5ddb4c1113f1e9fd74442646cb1b11dc6b"}
Sep 06 05:13:13.253529 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:13.253491    2576 patch_prober.go:28] interesting pod/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 06 05:13:13.253975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:13.253548    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 06 05:13:13.861374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:13.861335    2576 generic.go:356] "Generic (PLEG): container finished" podID="97459442-f194-4b5d-a09a-8f09632a4603" containerID="fd9c6e111f1d030f9603209d2ea33d067f708aea467f09eb75466c673cb4e4f4" exitCode=0
Sep 06 05:13:13.861564 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:13.861399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q" event={"ID":"97459442-f194-4b5d-a09a-8f09632a4603","Type":"ContainerDied","Data":"fd9c6e111f1d030f9603209d2ea33d067f708aea467f09eb75466c673cb4e4f4"}
Sep 06 05:13:14.257726 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:14.257695    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:14.258233 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:14.258119    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:14.258959 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:14.258939    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:15.004641 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.004613    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:15.053038 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.053003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca\") pod \"97459442-f194-4b5d-a09a-8f09632a4603\" (UID: \"97459442-f194-4b5d-a09a-8f09632a4603\") "
Sep 06 05:13:15.053385 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.053357    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "97459442-f194-4b5d-a09a-8f09632a4603" (UID: "97459442-f194-4b5d-a09a-8f09632a4603"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:15.154527 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.154439    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/97459442-f194-4b5d-a09a-8f09632a4603-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:15.872159 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.872131    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"
Sep 06 05:13:15.872571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.872164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q" event={"ID":"97459442-f194-4b5d-a09a-8f09632a4603","Type":"ContainerDied","Data":"2c31f801362519586ab4f3fee76a2b5ddb4c1113f1e9fd74442646cb1b11dc6b"}
Sep 06 05:13:15.872571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:15.872217    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c31f801362519586ab4f3fee76a2b5ddb4c1113f1e9fd74442646cb1b11dc6b"
Sep 06 05:13:18.641007 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.640975    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:13:18.641725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.641695    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" podUID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" containerName="payload-processing" containerID="cri-o://cb6db4fa92d27fc72337e4bf3951b9a91ab97ee9aec130ed3c487c9200b86f4b" gracePeriod=30
Sep 06 05:13:18.784129 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.784093    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"]
Sep 06 05:13:18.788820 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.788792    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-de5ebc7b-cvr7q"]
Sep 06 05:13:18.893781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.893692    2576 generic.go:356] "Generic (PLEG): container finished" podID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" containerID="cb6db4fa92d27fc72337e4bf3951b9a91ab97ee9aec130ed3c487c9200b86f4b" exitCode=0
Sep 06 05:13:18.893992 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.893886    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" event={"ID":"a8d27a57-4b57-43ca-bc60-748a7c69bc2b","Type":"ContainerDied","Data":"cb6db4fa92d27fc72337e4bf3951b9a91ab97ee9aec130ed3c487c9200b86f4b"}
Sep 06 05:13:18.893992 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.893984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph" event={"ID":"a8d27a57-4b57-43ca-bc60-748a7c69bc2b","Type":"ContainerDied","Data":"26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1"}
Sep 06 05:13:18.894129 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.894013    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26a5206a534f8b6fb94f260526f699d41c90858c84942399086698ff7c3ba4b1"
Sep 06 05:13:18.901623 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.901251    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:13:18.911562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.911535    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:13:18.911860 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.911799    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy" containerID="cri-o://2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4" gracePeriod=30
Sep 06 05:13:18.991652 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.991617    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume\") pod \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") "
Sep 06 05:13:18.991849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.991733    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwmq5\" (UniqueName: \"kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5\") pod \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\" (UID: \"a8d27a57-4b57-43ca-bc60-748a7c69bc2b\") "
Sep 06 05:13:18.992108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.992082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8d27a57-4b57-43ca-bc60-748a7c69bc2b" (UID: "a8d27a57-4b57-43ca-bc60-748a7c69bc2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:18.993943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:18.993921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5" (OuterVolumeSpecName: "kube-api-access-pwmq5") pod "a8d27a57-4b57-43ca-bc60-748a7c69bc2b" (UID: "a8d27a57-4b57-43ca-bc60-748a7c69bc2b"). InnerVolumeSpecName "kube-api-access-pwmq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:19.092622 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:19.092583    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwmq5\" (UniqueName: \"kubernetes.io/projected/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-kube-api-access-pwmq5\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:19.092622 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:19.092617    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d27a57-4b57-43ca-bc60-748a7c69bc2b-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:19.898963 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:19.898934    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"
Sep 06 05:13:19.919710 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:19.919684    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:13:19.925729 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:19.925707    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-de5ebc7b-84789865d9-55nph"]
Sep 06 05:13:20.386382 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:20.386336    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97459442-f194-4b5d-a09a-8f09632a4603" path="/var/lib/kubelet/pods/97459442-f194-4b5d-a09a-8f09632a4603/volumes"
Sep 06 05:13:20.387028 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:20.387008    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" path="/var/lib/kubelet/pods/a8d27a57-4b57-43ca-bc60-748a7c69bc2b/volumes"
Sep 06 05:13:24.169615 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.169589    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:24.244315 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244230    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c92v\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244315 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244262    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244315 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244303    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244330    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244423    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244459    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244511    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244550    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244634    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy\") pod \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\" (UID: \"e4adf83e-ad8e-460f-92e5-7beb387b49d2\") "
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244630    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244741    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244789    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:24.244872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.244833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:24.245239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.245011    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.245239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.245030    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.245239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.245042    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.245239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.245053    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.245239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.245064    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.246833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.246807    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:24.246981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.246871    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v" (OuterVolumeSpecName: "kube-api-access-9c92v") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "kube-api-access-9c92v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:24.247087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.247068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:13:24.247454 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.247426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4adf83e-ad8e-460f-92e5-7beb387b49d2" (UID: "e4adf83e-ad8e-460f-92e5-7beb387b49d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:13:24.345673 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.345640    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.345673 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.345671    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c92v\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-kube-api-access-9c92v\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.345890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.345684    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.345890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.345695    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4adf83e-ad8e-460f-92e5-7beb387b49d2-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:24.924241 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.924199    2576 generic.go:356] "Generic (PLEG): container finished" podID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerID="2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4" exitCode=0
Sep 06 05:13:24.924413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.924262    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"
Sep 06 05:13:24.924413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.924280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerDied","Data":"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"}
Sep 06 05:13:24.924413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.924346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf" event={"ID":"e4adf83e-ad8e-460f-92e5-7beb387b49d2","Type":"ContainerDied","Data":"17a066915fa4a21c2feabb56a22a80fda600b65192bffba982f0e71bef09b35b"}
Sep 06 05:13:24.924413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.924377    2576 scope.go:117] "RemoveContainer" containerID="2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"
Sep 06 05:13:24.933172 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.933148    2576 scope.go:117] "RemoveContainer" containerID="9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"
Sep 06 05:13:24.942393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.942373    2576 scope.go:117] "RemoveContainer" containerID="2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"
Sep 06 05:13:24.942809 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:13:24.942774    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4\": container with ID starting with 2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4 not found: ID does not exist" containerID="2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"
Sep 06 05:13:24.942926 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.942822    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:13:24.942926 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.942821    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4"} err="failed to get container status \"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4\": rpc error: code = NotFound desc = could not find container \"2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4\": container with ID starting with 2c5464558697e871e6af0589fc730c813757157ae2f6e836eb60b5d90133bad4 not found: ID does not exist"
Sep 06 05:13:24.942926 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.942876    2576 scope.go:117] "RemoveContainer" containerID="9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"
Sep 06 05:13:24.943376 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:13:24.943336    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331\": container with ID starting with 9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331 not found: ID does not exist" containerID="9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"
Sep 06 05:13:24.943438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.943373    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331"} err="failed to get container status \"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331\": rpc error: code = NotFound desc = could not find container \"9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331\": container with ID starting with 9c655de6d79867184220d4e92cc4545e2c239c845112dab2fe0ee57f75b36331 not found: ID does not exist"
Sep 06 05:13:24.944868 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:24.944850    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-de5ebc7b-openshift-default-695b4d6568-xcwvf"]
Sep 06 05:13:26.386371 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:26.386338    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" path="/var/lib/kubelet/pods/e4adf83e-ad8e-460f-92e5-7beb387b49d2/volumes"
Sep 06 05:13:28.091716 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.091677    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"]
Sep 06 05:13:28.092133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092103    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97459442-f194-4b5d-a09a-8f09632a4603" containerName="revoke-keys"
Sep 06 05:13:28.092133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092115    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="97459442-f194-4b5d-a09a-8f09632a4603" containerName="revoke-keys"
Sep 06 05:13:28.092133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092128    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092134    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092143    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092149    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" containerName="payload-processing"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092164    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" containerName="payload-processing"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092253    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="97459442-f194-4b5d-a09a-8f09632a4603" containerName="revoke-keys"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092265    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092278    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4adf83e-ad8e-460f-92e5-7beb387b49d2" containerName="istio-proxy"
Sep 06 05:13:28.092331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.092288    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d27a57-4b57-43ca-bc60-748a7c69bc2b" containerName="payload-processing"
Sep 06 05:13:28.095363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.095347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:28.097814 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.097792    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:13:28.102313 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.102293    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"]
Sep 06 05:13:28.184692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.184656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9afd7f-sz67d\" (UID: \"a48796b8-69c9-4d60-a1d4-3b8a71e09363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:28.285885 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.285842    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9afd7f-sz67d\" (UID: \"a48796b8-69c9-4d60-a1d4-3b8a71e09363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:28.286551 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.286525    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-9afd7f-sz67d\" (UID: \"a48796b8-69c9-4d60-a1d4-3b8a71e09363\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:28.408406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.408315    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:28.680082 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.680056    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"]
Sep 06 05:13:28.682121 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:13:28.682095    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48796b8_69c9_4d60_a1d4_3b8a71e09363.slice/crio-185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5 WatchSource:0}: Error finding container 185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5: Status 404 returned error can't find the container with id 185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5
Sep 06 05:13:28.950252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:28.950160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d" event={"ID":"a48796b8-69c9-4d60-a1d4-3b8a71e09363","Type":"ContainerStarted","Data":"185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5"}
Sep 06 05:13:29.956827 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:29.956795    2576 generic.go:356] "Generic (PLEG): container finished" podID="a48796b8-69c9-4d60-a1d4-3b8a71e09363" containerID="0fa595ae233039ee914e0b2060a98176e4af0ffd69802ec36d6dcb2e35ad4bcb" exitCode=0
Sep 06 05:13:29.957227 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:29.956885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d" event={"ID":"a48796b8-69c9-4d60-a1d4-3b8a71e09363","Type":"ContainerDied","Data":"0fa595ae233039ee914e0b2060a98176e4af0ffd69802ec36d6dcb2e35ad4bcb"}
Sep 06 05:13:31.094823 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.094800    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:31.214061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.213973    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca\") pod \"a48796b8-69c9-4d60-a1d4-3b8a71e09363\" (UID: \"a48796b8-69c9-4d60-a1d4-3b8a71e09363\") "
Sep 06 05:13:31.214333 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.214307    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a48796b8-69c9-4d60-a1d4-3b8a71e09363" (UID: "a48796b8-69c9-4d60-a1d4-3b8a71e09363"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:31.314932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.314881    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48796b8-69c9-4d60-a1d4-3b8a71e09363-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:31.968495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.968463    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"
Sep 06 05:13:31.968699 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.968498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d" event={"ID":"a48796b8-69c9-4d60-a1d4-3b8a71e09363","Type":"ContainerDied","Data":"185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5"}
Sep 06 05:13:31.968699 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:31.968552    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="185fa51a0eb58ae5e1e719cb8a60e31b6ba11473ebc80ef2c84fe780073dbbe5"
Sep 06 05:13:38.551932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.551883    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:13:38.552406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.552110    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" podUID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" containerName="payload-processing" containerID="cri-o://1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067" gracePeriod=30
Sep 06 05:13:38.560198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.559762    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:13:38.693962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.693862    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"]
Sep 06 05:13:38.697117 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.697089    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-9afd7f-sz67d"]
Sep 06 05:13:38.819975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.819946    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:13:38.987196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.987101    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume\") pod \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") "
Sep 06 05:13:38.987359 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.987273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvmmz\" (UniqueName: \"kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz\") pod \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\" (UID: \"84b36caf-ceaa-4ff3-85d1-1342363a7bfa\") "
Sep 06 05:13:38.987558 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.987537    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84b36caf-ceaa-4ff3-85d1-1342363a7bfa" (UID: "84b36caf-ceaa-4ff3-85d1-1342363a7bfa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:38.989668 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:38.989642    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz" (OuterVolumeSpecName: "kube-api-access-nvmmz") pod "84b36caf-ceaa-4ff3-85d1-1342363a7bfa" (UID: "84b36caf-ceaa-4ff3-85d1-1342363a7bfa"). InnerVolumeSpecName "kube-api-access-nvmmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:39.008507 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008473    2576 generic.go:356] "Generic (PLEG): container finished" podID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" containerID="1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067" exitCode=0
Sep 06 05:13:39.008675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008522    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"
Sep 06 05:13:39.008675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" event={"ID":"84b36caf-ceaa-4ff3-85d1-1342363a7bfa","Type":"ContainerDied","Data":"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"}
Sep 06 05:13:39.008675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn" event={"ID":"84b36caf-ceaa-4ff3-85d1-1342363a7bfa","Type":"ContainerDied","Data":"9090a904203c98b17b60638f19ebc10d97d1954ced4bb4cbd1f94b26c1a9e175"}
Sep 06 05:13:39.008675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008659    2576 scope.go:117] "RemoveContainer" containerID="1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"
Sep 06 05:13:39.008994 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.008959    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" podUID="807ac03e-8c4e-4e93-8560-d34d05a734ad" containerName="payload-pre-processing" containerID="cri-o://9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089" gracePeriod=30
Sep 06 05:13:39.019217 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.019198    2576 scope.go:117] "RemoveContainer" containerID="1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"
Sep 06 05:13:39.019516 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:13:39.019498    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067\": container with ID starting with 1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067 not found: ID does not exist" containerID="1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"
Sep 06 05:13:39.019579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.019527    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067"} err="failed to get container status \"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067\": rpc error: code = NotFound desc = could not find container \"1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067\": container with ID starting with 1c53956cf1d9c02d83759bf977c966d7d67b9cabd29fadcfb4cc5f570279d067 not found: ID does not exist"
Sep 06 05:13:39.039520 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.039486    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:13:39.041892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.041866    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-9afd7f-5f769c74f9-fq5bn"]
Sep 06 05:13:39.088357 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.088323    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:39.088357 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.088353    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvmmz\" (UniqueName: \"kubernetes.io/projected/84b36caf-ceaa-4ff3-85d1-1342363a7bfa-kube-api-access-nvmmz\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:39.253079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.253056    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:13:39.391777 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.391741    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnfz2\" (UniqueName: \"kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2\") pod \"807ac03e-8c4e-4e93-8560-d34d05a734ad\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") "
Sep 06 05:13:39.392018 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.391797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume\") pod \"807ac03e-8c4e-4e93-8560-d34d05a734ad\" (UID: \"807ac03e-8c4e-4e93-8560-d34d05a734ad\") "
Sep 06 05:13:39.392284 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.392255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "807ac03e-8c4e-4e93-8560-d34d05a734ad" (UID: "807ac03e-8c4e-4e93-8560-d34d05a734ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:13:39.394161 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.394141    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2" (OuterVolumeSpecName: "kube-api-access-pnfz2") pod "807ac03e-8c4e-4e93-8560-d34d05a734ad" (UID: "807ac03e-8c4e-4e93-8560-d34d05a734ad"). InnerVolumeSpecName "kube-api-access-pnfz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:13:39.492713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.492668    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/807ac03e-8c4e-4e93-8560-d34d05a734ad-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:39.492713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:39.492701    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnfz2\" (UniqueName: \"kubernetes.io/projected/807ac03e-8c4e-4e93-8560-d34d05a734ad-kube-api-access-pnfz2\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:13:40.014718 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.014669    2576 generic.go:356] "Generic (PLEG): container finished" podID="807ac03e-8c4e-4e93-8560-d34d05a734ad" containerID="9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089" exitCode=0
Sep 06 05:13:40.014718 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.014718    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"
Sep 06 05:13:40.015252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.014763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" event={"ID":"807ac03e-8c4e-4e93-8560-d34d05a734ad","Type":"ContainerDied","Data":"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"}
Sep 06 05:13:40.015252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.014813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2" event={"ID":"807ac03e-8c4e-4e93-8560-d34d05a734ad","Type":"ContainerDied","Data":"33380819b19a39b2af6726005ebc08016ac028f2ba80a9e11d48c791373adf1d"}
Sep 06 05:13:40.015252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.014851    2576 scope.go:117] "RemoveContainer" containerID="9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"
Sep 06 05:13:40.023937 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.023895    2576 scope.go:117] "RemoveContainer" containerID="9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"
Sep 06 05:13:40.024221 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:13:40.024191    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089\": container with ID starting with 9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089 not found: ID does not exist" containerID="9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"
Sep 06 05:13:40.024272 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.024231    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089"} err="failed to get container status \"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089\": rpc error: code = NotFound desc = could not find container \"9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089\": container with ID starting with 9c3100b7b77f4e48eb08a024ce6e11eed4f429b5f744a81aa81ec63d5a3d1089 not found: ID does not exist"
Sep 06 05:13:40.037455 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.037428    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:13:40.040723 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.040701    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-9afd7f-5cbff48b8b-vh8r2"]
Sep 06 05:13:40.386708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.386615    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="807ac03e-8c4e-4e93-8560-d34d05a734ad" path="/var/lib/kubelet/pods/807ac03e-8c4e-4e93-8560-d34d05a734ad/volumes"
Sep 06 05:13:40.387108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.387088    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" path="/var/lib/kubelet/pods/84b36caf-ceaa-4ff3-85d1-1342363a7bfa/volumes"
Sep 06 05:13:40.387428 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:40.387411    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48796b8-69c9-4d60-a1d4-3b8a71e09363" path="/var/lib/kubelet/pods/a48796b8-69c9-4d60-a1d4-3b8a71e09363/volumes"
Sep 06 05:13:50.065886 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.065855    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066284    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" containerName="payload-processing"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066298    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" containerName="payload-processing"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066318    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807ac03e-8c4e-4e93-8560-d34d05a734ad" containerName="payload-pre-processing"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066325    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="807ac03e-8c4e-4e93-8560-d34d05a734ad" containerName="payload-pre-processing"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066337    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48796b8-69c9-4d60-a1d4-3b8a71e09363" containerName="revoke-keys"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066342    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48796b8-69c9-4d60-a1d4-3b8a71e09363" containerName="revoke-keys"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066403    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48796b8-69c9-4d60-a1d4-3b8a71e09363" containerName="revoke-keys"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066410    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84b36caf-ceaa-4ff3-85d1-1342363a7bfa" containerName="payload-processing"
Sep 06 05:13:50.066412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.066418    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="807ac03e-8c4e-4e93-8560-d34d05a734ad" containerName="payload-pre-processing"
Sep 06 05:13:50.071141 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.071114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.074018 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.073996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-17d595-dockercfg-qn9w8\""
Sep 06 05:13:50.074117 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.074044    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-17d595\""
Sep 06 05:13:50.080448 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.080423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:13:50.092318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.091478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stj6x\" (UniqueName: \"kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.092318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.091543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.104450 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.104425    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:13:50.107911 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.107883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.115079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.115029    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:13:50.192722 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.192675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stj6x\" (UniqueName: \"kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.192932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.192730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4znm\" (UniqueName: \"kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.192932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.192764    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.192932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.192799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.193699 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.193675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.201151 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.201127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stj6x\" (UniqueName: \"kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x\") pod \"payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.293321 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.293286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.293500 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.293402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4znm\" (UniqueName: \"kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.293987 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.293968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.303669 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.303639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4znm\" (UniqueName: \"kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm\") pod \"payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") " pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.382835 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.382749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:50.419467 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.419427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:50.659304 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.659277    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:13:50.661459 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:13:50.661424    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fccd1db_06b5_4476_bfa7_010a64b47f1d.slice/crio-012ca859994b75e1120f35799b46bca65c65cd102a3dbe9d7b1f59ad511028e0 WatchSource:0}: Error finding container 012ca859994b75e1120f35799b46bca65c65cd102a3dbe9d7b1f59ad511028e0: Status 404 returned error can't find the container with id 012ca859994b75e1120f35799b46bca65c65cd102a3dbe9d7b1f59ad511028e0
Sep 06 05:13:50.701933 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:50.701893    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:13:50.716071 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:13:50.716048    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f3e06d_c11f_4a88_9247_971acf132112.slice/crio-4adbe7abc21c55f0d2cd867aaf538263a08f1b93eb3d1e856cda902267293ead WatchSource:0}: Error finding container 4adbe7abc21c55f0d2cd867aaf538263a08f1b93eb3d1e856cda902267293ead: Status 404 returned error can't find the container with id 4adbe7abc21c55f0d2cd867aaf538263a08f1b93eb3d1e856cda902267293ead
Sep 06 05:13:51.076198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.076158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" event={"ID":"98f3e06d-c11f-4a88-9247-971acf132112","Type":"ContainerStarted","Data":"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"}
Sep 06 05:13:51.076737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.076229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" event={"ID":"98f3e06d-c11f-4a88-9247-971acf132112","Type":"ContainerStarted","Data":"4adbe7abc21c55f0d2cd867aaf538263a08f1b93eb3d1e856cda902267293ead"}
Sep 06 05:13:51.076737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.076275    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:13:51.077991 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.077965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" event={"ID":"6fccd1db-06b5-4476-bfa7-010a64b47f1d","Type":"ContainerStarted","Data":"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"}
Sep 06 05:13:51.078125 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.078011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" event={"ID":"6fccd1db-06b5-4476-bfa7-010a64b47f1d","Type":"ContainerStarted","Data":"012ca859994b75e1120f35799b46bca65c65cd102a3dbe9d7b1f59ad511028e0"}
Sep 06 05:13:51.078125 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.078098    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:13:51.091409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.091350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" podStartSLOduration=1.091326831 podStartE2EDuration="1.091326831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:13:51.091206879 +0000 UTC m=+1093.180284577" watchObservedRunningTime="2026-09-06 05:13:51.091326831 +0000 UTC m=+1093.180404526"
Sep 06 05:13:51.105895 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:13:51.105845    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" podStartSLOduration=1.1058276870000001 podStartE2EDuration="1.105827687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:13:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:13:51.105227601 +0000 UTC m=+1093.194305293" watchObservedRunningTime="2026-09-06 05:13:51.105827687 +0000 UTC m=+1093.194905380"
Sep 06 05:14:02.084870 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:02.084838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:14:02.085407 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:02.084923    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:14:58.444484 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.444449    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:14:58.449253 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.449236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.451747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.451724    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-241a3769-dockercfg-gd2dr\""
Sep 06 05:14:58.451857 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.451766    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-241a3769\""
Sep 06 05:14:58.458385 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.458361    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:14:58.479593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.479556    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:14:58.483863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.483838    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.490224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.490198    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:14:58.524452 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.524421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp5wh\" (UniqueName: \"kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.524593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.524513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.625339 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.625296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44p22\" (UniqueName: \"kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.625562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.625377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp5wh\" (UniqueName: \"kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.625562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.625463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.625562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.625498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.626538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.626511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.633367 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.633337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp5wh\" (UniqueName: \"kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh\") pod \"payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.726287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.726188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.726287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.726230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44p22\" (UniqueName: \"kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.727102 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.727080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.733499 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.733471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44p22\" (UniqueName: \"kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22\") pod \"payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") " pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:58.760561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.760538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:58.796980 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:58.796950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:59.030979 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.030955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:14:59.032815 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:14:59.032788    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46820a40_76ad_4299_aea4_c641cbeed911.slice/crio-c3cbd3d79ada2e7f15f324a5148f61d602aca12afe4feacf7d5313cfe38e7cc9 WatchSource:0}: Error finding container c3cbd3d79ada2e7f15f324a5148f61d602aca12afe4feacf7d5313cfe38e7cc9: Status 404 returned error can't find the container with id c3cbd3d79ada2e7f15f324a5148f61d602aca12afe4feacf7d5313cfe38e7cc9
Sep 06 05:14:59.079053 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.079031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:14:59.081298 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:14:59.081276    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad57e2e_992b_4a15_a187_a2dfe2133d19.slice/crio-377fbbc245c8f59d0452c13a9b38cfaf141be7f5b1b2ac0c37e4d25169fb10c7 WatchSource:0}: Error finding container 377fbbc245c8f59d0452c13a9b38cfaf141be7f5b1b2ac0c37e4d25169fb10c7: Status 404 returned error can't find the container with id 377fbbc245c8f59d0452c13a9b38cfaf141be7f5b1b2ac0c37e4d25169fb10c7
Sep 06 05:14:59.420288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.420165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" event={"ID":"bad57e2e-992b-4a15-a187-a2dfe2133d19","Type":"ContainerStarted","Data":"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"}
Sep 06 05:14:59.420288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.420230    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" event={"ID":"bad57e2e-992b-4a15-a187-a2dfe2133d19","Type":"ContainerStarted","Data":"377fbbc245c8f59d0452c13a9b38cfaf141be7f5b1b2ac0c37e4d25169fb10c7"}
Sep 06 05:14:59.420505 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.420327    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:14:59.421863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.421839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" event={"ID":"46820a40-76ad-4299-aea4-c641cbeed911","Type":"ContainerStarted","Data":"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"}
Sep 06 05:14:59.422011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.421880    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:14:59.422011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.421914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" event={"ID":"46820a40-76ad-4299-aea4-c641cbeed911","Type":"ContainerStarted","Data":"c3cbd3d79ada2e7f15f324a5148f61d602aca12afe4feacf7d5313cfe38e7cc9"}
Sep 06 05:14:59.436183 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.436146    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" podStartSLOduration=1.436134187 podStartE2EDuration="1.436134187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:14:59.434334363 +0000 UTC m=+1161.523412137" watchObservedRunningTime="2026-09-06 05:14:59.436134187 +0000 UTC m=+1161.525211879"
Sep 06 05:14:59.448858 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:14:59.448819    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" podStartSLOduration=1.4488077160000001 podStartE2EDuration="1.448807716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:14:59.447543956 +0000 UTC m=+1161.536621649" watchObservedRunningTime="2026-09-06 05:14:59.448807716 +0000 UTC m=+1161.537885406"
Sep 06 05:15:00.133156 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.133122    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"]
Sep 06 05:15:00.136837 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.136816    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"]
Sep 06 05:15:00.136982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.136947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"
Sep 06 05:15:00.139207 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.139189    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:15:00.140107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.140089    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"]
Sep 06 05:15:00.140200 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.140187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"
Sep 06 05:15:00.143509 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.143493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"
Sep 06 05:15:00.143711 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.143692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"]
Sep 06 05:15:00.147241 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.147220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"]
Sep 06 05:15:00.152021 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.149661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"]
Sep 06 05:15:00.152021 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.151845    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"
Sep 06 05:15:00.162751 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.162727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"
Sep 06 05:15:00.203454 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.203427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"
Sep 06 05:15:00.439228 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.439203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"]
Sep 06 05:15:00.441129 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:00.441105    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6532632_9fee_48f3_9fca_2f7af9526aa4.slice/crio-a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad WatchSource:0}: Error finding container a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad: Status 404 returned error can't find the container with id a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad
Sep 06 05:15:00.441638 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.441539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"]
Sep 06 05:15:00.442268 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:00.442245    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6ba1ea_127f_4a69_a833_b27796052ded.slice/crio-75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1 WatchSource:0}: Error finding container 75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1: Status 404 returned error can't find the container with id 75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1
Sep 06 05:15:00.493856 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:00.493834    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"]
Sep 06 05:15:00.495722 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:00.495693    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2198a08a_1d2b_4fb3_8df9_9bcbc4b7ef71.slice/crio-b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f WatchSource:0}: Error finding container b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f: Status 404 returned error can't find the container with id b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f
Sep 06 05:15:01.433506 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.433473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745" event={"ID":"2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71","Type":"ContainerStarted","Data":"7759239492cfdba724401f05e4732b5eba4ea194ea7086c0ba360dc9d2c6ee81"}
Sep 06 05:15:01.433694 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.433528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745" event={"ID":"2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71","Type":"ContainerStarted","Data":"b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f"}
Sep 06 05:15:01.435178 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.435153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc" event={"ID":"e6532632-9fee-48f3-9fca-2f7af9526aa4","Type":"ContainerStarted","Data":"19f9494750b446d79cc20a336259b8028266aa028347724a068900170c85372f"}
Sep 06 05:15:01.435318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.435201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc" event={"ID":"e6532632-9fee-48f3-9fca-2f7af9526aa4","Type":"ContainerStarted","Data":"a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad"}
Sep 06 05:15:01.436671 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.436645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg" event={"ID":"1c6ba1ea-127f-4a69-a833-b27796052ded","Type":"ContainerStarted","Data":"2d48ee21410c5f4515425d267c6e6791dd2a966b2c20f9c536c5c74551028865"}
Sep 06 05:15:01.436754 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.436686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg" event={"ID":"1c6ba1ea-127f-4a69-a833-b27796052ded","Type":"ContainerStarted","Data":"75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1"}
Sep 06 05:15:01.447597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.447543    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745" podStartSLOduration=1.44752677 podStartE2EDuration="1.44752677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:01.446201741 +0000 UTC m=+1163.535279434" watchObservedRunningTime="2026-09-06 05:15:01.44752677 +0000 UTC m=+1163.536604455"
Sep 06 05:15:01.459835 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.459788    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc" podStartSLOduration=1.459774069 podStartE2EDuration="1.459774069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:01.458111585 +0000 UTC m=+1163.547189280" watchObservedRunningTime="2026-09-06 05:15:01.459774069 +0000 UTC m=+1163.548851762"
Sep 06 05:15:01.471988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:01.471942    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg" podStartSLOduration=1.4719284830000001 podStartE2EDuration="1.471928483s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:01.47094468 +0000 UTC m=+1163.560022374" watchObservedRunningTime="2026-09-06 05:15:01.471928483 +0000 UTC m=+1163.561006175"
Sep 06 05:15:02.444773 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.444737    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c6ba1ea-127f-4a69-a833-b27796052ded" containerID="2d48ee21410c5f4515425d267c6e6791dd2a966b2c20f9c536c5c74551028865" exitCode=0
Sep 06 05:15:02.445282 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.444843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg" event={"ID":"1c6ba1ea-127f-4a69-a833-b27796052ded","Type":"ContainerDied","Data":"2d48ee21410c5f4515425d267c6e6791dd2a966b2c20f9c536c5c74551028865"}
Sep 06 05:15:02.447037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.447012    2576 generic.go:356] "Generic (PLEG): container finished" podID="2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71" containerID="7759239492cfdba724401f05e4732b5eba4ea194ea7086c0ba360dc9d2c6ee81" exitCode=0
Sep 06 05:15:02.447171 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.447041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745" event={"ID":"2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71","Type":"ContainerDied","Data":"7759239492cfdba724401f05e4732b5eba4ea194ea7086c0ba360dc9d2c6ee81"}
Sep 06 05:15:02.449029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.449008    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6532632-9fee-48f3-9fca-2f7af9526aa4" containerID="19f9494750b446d79cc20a336259b8028266aa028347724a068900170c85372f" exitCode=0
Sep 06 05:15:02.449153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:02.449085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc" event={"ID":"e6532632-9fee-48f3-9fca-2f7af9526aa4","Type":"ContainerDied","Data":"19f9494750b446d79cc20a336259b8028266aa028347724a068900170c85372f"}
Sep 06 05:15:03.599021 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:03.598998    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"
Sep 06 05:15:03.646737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:03.646717    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"
Sep 06 05:15:03.673219 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:03.673189    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"
Sep 06 05:15:04.461256 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.461225    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"
Sep 06 05:15:04.461256 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.461228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745" event={"ID":"2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71","Type":"ContainerDied","Data":"b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f"}
Sep 06 05:15:04.461487 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.461292    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8b72fc68110ba6c0351f501019ea8a9f54ccc66fc98206f2636f2c5fe95b26f"
Sep 06 05:15:04.462626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.462604    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc"
Sep 06 05:15:04.462755 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.462647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811195-djjfc" event={"ID":"e6532632-9fee-48f3-9fca-2f7af9526aa4","Type":"ContainerDied","Data":"a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad"}
Sep 06 05:15:04.462755 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.462691    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8c009198fb83cfddac94317adff8e330ed35521c24830b256a920dc70dea8ad"
Sep 06 05:15:04.464121 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.464095    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg" event={"ID":"1c6ba1ea-127f-4a69-a833-b27796052ded","Type":"ContainerDied","Data":"75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1"}
Sep 06 05:15:04.464239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.464142    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75d1ac326f8297c0c354dce72dbdb1f54b8027f9d69f27deae27cbe3ea6a61e1"
Sep 06 05:15:04.464239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:04.464107    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"
Sep 06 05:15:05.664017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.663990    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"]
Sep 06 05:15:05.664392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6532632-9fee-48f3-9fca-2f7af9526aa4" containerName="cleanup"
Sep 06 05:15:05.664437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664394    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6532632-9fee-48f3-9fca-2f7af9526aa4" containerName="cleanup"
Sep 06 05:15:05.664437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664404    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6ba1ea-127f-4a69-a833-b27796052ded" containerName="cleanup"
Sep 06 05:15:05.664437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664411    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6ba1ea-127f-4a69-a833-b27796052ded" containerName="cleanup"
Sep 06 05:15:05.664437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664431    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71" containerName="cleanup"
Sep 06 05:15:05.664437 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664436    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71" containerName="cleanup"
Sep 06 05:15:05.664593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664511    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c6ba1ea-127f-4a69-a833-b27796052ded" containerName="cleanup"
Sep 06 05:15:05.664593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664520    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6532632-9fee-48f3-9fca-2f7af9526aa4" containerName="cleanup"
Sep 06 05:15:05.664593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.664527    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71" containerName="cleanup"
Sep 06 05:15:05.667517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.667501    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:05.669911 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.669877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:15:05.673969 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.673944    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"]
Sep 06 05:15:05.794791 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.794757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-17d595-hwsvn\" (UID: \"5e6526e9-d669-4c2f-89c8-aea7073e501a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:05.896291 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.896252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-17d595-hwsvn\" (UID: \"5e6526e9-d669-4c2f-89c8-aea7073e501a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:05.896921 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.896883    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-17d595-hwsvn\" (UID: \"5e6526e9-d669-4c2f-89c8-aea7073e501a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:05.979801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:05.979767    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:06.254539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:06.254516    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"]
Sep 06 05:15:06.256668 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:06.256641    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e6526e9_d669_4c2f_89c8_aea7073e501a.slice/crio-ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb WatchSource:0}: Error finding container ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb: Status 404 returned error can't find the container with id ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb
Sep 06 05:15:06.476095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:06.476067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn" event={"ID":"5e6526e9-d669-4c2f-89c8-aea7073e501a","Type":"ContainerStarted","Data":"ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb"}
Sep 06 05:15:07.484741 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:07.484699    2576 generic.go:356] "Generic (PLEG): container finished" podID="5e6526e9-d669-4c2f-89c8-aea7073e501a" containerID="254afba5fa69c922b6914546618f28bdafc16569f6952e6e2cf293e334215b3f" exitCode=0
Sep 06 05:15:07.485153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:07.484777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn" event={"ID":"5e6526e9-d669-4c2f-89c8-aea7073e501a","Type":"ContainerDied","Data":"254afba5fa69c922b6914546618f28bdafc16569f6952e6e2cf293e334215b3f"}
Sep 06 05:15:08.629975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:08.629951    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:08.722220 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:08.722187    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca\") pod \"5e6526e9-d669-4c2f-89c8-aea7073e501a\" (UID: \"5e6526e9-d669-4c2f-89c8-aea7073e501a\") "
Sep 06 05:15:08.722531 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:08.722510    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e6526e9-d669-4c2f-89c8-aea7073e501a" (UID: "5e6526e9-d669-4c2f-89c8-aea7073e501a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:08.823409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:08.823326    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e6526e9-d669-4c2f-89c8-aea7073e501a-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:09.496770 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:09.496740    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"
Sep 06 05:15:09.497037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:09.496739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn" event={"ID":"5e6526e9-d669-4c2f-89c8-aea7073e501a","Type":"ContainerDied","Data":"ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb"}
Sep 06 05:15:09.497037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:09.496866    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed17f3fcbe91be2bc27dfca2b7359429a4dbd2773d593fce16adfc2a6fb8e0fb"
Sep 06 05:15:10.429038 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:10.429010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:15:10.429543 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:10.429092    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:15:16.111834 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.111803    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"]
Sep 06 05:15:16.113672 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.113652    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-17d595-29811195-wt745"]
Sep 06 05:15:16.133788 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.133751    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:15:16.134091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.134039    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" podUID="98f3e06d-c11f-4a88-9247-971acf132112" containerName="payload-processing" containerID="cri-o://5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b" gracePeriod=30
Sep 06 05:15:16.147094 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.147062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:15:16.147338 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.147302    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" podUID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" containerName="payload-pre-processing" containerID="cri-o://584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349" gracePeriod=30
Sep 06 05:15:16.296930 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.296866    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"]
Sep 06 05:15:16.298892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.298859    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-17d595-hwsvn"]
Sep 06 05:15:16.387178 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.387099    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71" path="/var/lib/kubelet/pods/2198a08a-1d2b-4fb3-8df9-9bcbc4b7ef71/volumes"
Sep 06 05:15:16.387452 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.387436    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e6526e9-d669-4c2f-89c8-aea7073e501a" path="/var/lib/kubelet/pods/5e6526e9-d669-4c2f-89c8-aea7073e501a/volumes"
Sep 06 05:15:16.428981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.428959    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:15:16.430974 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.430954    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:15:16.536032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.535999    2576 generic.go:356] "Generic (PLEG): container finished" podID="98f3e06d-c11f-4a88-9247-971acf132112" containerID="5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b" exitCode=0
Sep 06 05:15:16.536259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.536054    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"
Sep 06 05:15:16.536259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.536092    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" event={"ID":"98f3e06d-c11f-4a88-9247-971acf132112","Type":"ContainerDied","Data":"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"}
Sep 06 05:15:16.536259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.536156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5" event={"ID":"98f3e06d-c11f-4a88-9247-971acf132112","Type":"ContainerDied","Data":"4adbe7abc21c55f0d2cd867aaf538263a08f1b93eb3d1e856cda902267293ead"}
Sep 06 05:15:16.536259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.536184    2576 scope.go:117] "RemoveContainer" containerID="5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"
Sep 06 05:15:16.537508 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.537489    2576 generic.go:356] "Generic (PLEG): container finished" podID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" containerID="584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349" exitCode=0
Sep 06 05:15:16.537769 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.537525    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"
Sep 06 05:15:16.537769 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.537547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" event={"ID":"6fccd1db-06b5-4476-bfa7-010a64b47f1d","Type":"ContainerDied","Data":"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"}
Sep 06 05:15:16.537769 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.537596    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz" event={"ID":"6fccd1db-06b5-4476-bfa7-010a64b47f1d","Type":"ContainerDied","Data":"012ca859994b75e1120f35799b46bca65c65cd102a3dbe9d7b1f59ad511028e0"}
Sep 06 05:15:16.544892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.544876    2576 scope.go:117] "RemoveContainer" containerID="5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"
Sep 06 05:15:16.545192 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:15:16.545174    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b\": container with ID starting with 5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b not found: ID does not exist" containerID="5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"
Sep 06 05:15:16.545255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.545200    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b"} err="failed to get container status \"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b\": rpc error: code = NotFound desc = could not find container \"5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b\": container with ID starting with 5817811fef188254a462a3f934b3bcb0279f781ca33b3205d906ab10fb89432b not found: ID does not exist"
Sep 06 05:15:16.545255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.545219    2576 scope.go:117] "RemoveContainer" containerID="584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"
Sep 06 05:15:16.552993 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.552978    2576 scope.go:117] "RemoveContainer" containerID="584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"
Sep 06 05:15:16.553224 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:15:16.553205    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349\": container with ID starting with 584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349 not found: ID does not exist" containerID="584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"
Sep 06 05:15:16.553286 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.553229    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349"} err="failed to get container status \"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349\": rpc error: code = NotFound desc = could not find container \"584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349\": container with ID starting with 584b207f4f11fae3454dfdc42b0b84e67462085bfd88ba2e813315cfe0657349 not found: ID does not exist"
Sep 06 05:15:16.595849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.595822    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume\") pod \"98f3e06d-c11f-4a88-9247-971acf132112\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") "
Sep 06 05:15:16.596079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.595945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume\") pod \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") "
Sep 06 05:15:16.596079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.595985    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4znm\" (UniqueName: \"kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm\") pod \"98f3e06d-c11f-4a88-9247-971acf132112\" (UID: \"98f3e06d-c11f-4a88-9247-971acf132112\") "
Sep 06 05:15:16.596079 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.596008    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stj6x\" (UniqueName: \"kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x\") pod \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\" (UID: \"6fccd1db-06b5-4476-bfa7-010a64b47f1d\") "
Sep 06 05:15:16.596364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.596342    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fccd1db-06b5-4476-bfa7-010a64b47f1d" (UID: "6fccd1db-06b5-4476-bfa7-010a64b47f1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:16.596364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.596352    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98f3e06d-c11f-4a88-9247-971acf132112" (UID: "98f3e06d-c11f-4a88-9247-971acf132112"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:16.598277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.598255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm" (OuterVolumeSpecName: "kube-api-access-x4znm") pod "98f3e06d-c11f-4a88-9247-971acf132112" (UID: "98f3e06d-c11f-4a88-9247-971acf132112"). InnerVolumeSpecName "kube-api-access-x4znm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:16.598277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.598262    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x" (OuterVolumeSpecName: "kube-api-access-stj6x") pod "6fccd1db-06b5-4476-bfa7-010a64b47f1d" (UID: "6fccd1db-06b5-4476-bfa7-010a64b47f1d"). InnerVolumeSpecName "kube-api-access-stj6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:16.697708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.697675    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fccd1db-06b5-4476-bfa7-010a64b47f1d-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:16.697708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.697705    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4znm\" (UniqueName: \"kubernetes.io/projected/98f3e06d-c11f-4a88-9247-971acf132112-kube-api-access-x4znm\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:16.697708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.697714    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stj6x\" (UniqueName: \"kubernetes.io/projected/6fccd1db-06b5-4476-bfa7-010a64b47f1d-kube-api-access-stj6x\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:16.697947 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.697723    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98f3e06d-c11f-4a88-9247-971acf132112-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:16.860743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.860713    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:15:16.863876 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.863855    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-17d595-b7d4bdd74-64zwz"]
Sep 06 05:15:16.873601 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.873583    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:15:16.877311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:16.877288    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-17d595-774f55d9b9-tglt5"]
Sep 06 05:15:18.386168 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:18.386136    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" path="/var/lib/kubelet/pods/6fccd1db-06b5-4476-bfa7-010a64b47f1d/volumes"
Sep 06 05:15:18.386556 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:18.386500    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f3e06d-c11f-4a88-9247-971acf132112" path="/var/lib/kubelet/pods/98f3e06d-c11f-4a88-9247-971acf132112/volumes"
Sep 06 05:15:23.969098 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969052    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:23.969645 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969623    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" containerName="payload-pre-processing"
Sep 06 05:15:23.969734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969649    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" containerName="payload-pre-processing"
Sep 06 05:15:23.969734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969660    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f3e06d-c11f-4a88-9247-971acf132112" containerName="payload-processing"
Sep 06 05:15:23.969734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969668    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f3e06d-c11f-4a88-9247-971acf132112" containerName="payload-processing"
Sep 06 05:15:23.969734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969682    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e6526e9-d669-4c2f-89c8-aea7073e501a" containerName="revoke-keys"
Sep 06 05:15:23.969734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969693    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e6526e9-d669-4c2f-89c8-aea7073e501a" containerName="revoke-keys"
Sep 06 05:15:23.970003 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969787    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e6526e9-d669-4c2f-89c8-aea7073e501a" containerName="revoke-keys"
Sep 06 05:15:23.970003 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969801    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f3e06d-c11f-4a88-9247-971acf132112" containerName="payload-processing"
Sep 06 05:15:23.970003 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.969814    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fccd1db-06b5-4476-bfa7-010a64b47f1d" containerName="payload-pre-processing"
Sep 06 05:15:23.974704 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.974680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:23.977328 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.977303    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-c53189-openshift-default-dockercfg-6xd5w\""
Sep 06 05:15:23.983720 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:23.983696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:24.062962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.062923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.062962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.062968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063018    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pcx\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.063497 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.063322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.163966 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.163927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.163966 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.163967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pcx\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164644 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164644 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164644 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.164815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.164762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.166575 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.166558    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.166801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.166777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.171768 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.171747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.172113 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.172092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pcx\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx\") pod \"e2e-noten-c53189-openshift-default-75565d77f8-g4shq\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") " pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.287986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.287881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:24.567245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.567213    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:24.569279 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:24.569251    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b92348c_0d65_4080_be68_00562ca27791.slice/crio-c9a2290a800fad50d08506d1170b57fa3d459b5f1ec53b6607cc728a0767fdd5 WatchSource:0}: Error finding container c9a2290a800fad50d08506d1170b57fa3d459b5f1ec53b6607cc728a0767fdd5: Status 404 returned error can't find the container with id c9a2290a800fad50d08506d1170b57fa3d459b5f1ec53b6607cc728a0767fdd5
Sep 06 05:15:24.571839 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.571781    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:15:24.571989 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.571877    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:15:24.571989 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.571962    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:15:24.588068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:24.588032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" event={"ID":"3b92348c-0d65-4080-be68-00562ca27791","Type":"ContainerStarted","Data":"c9a2290a800fad50d08506d1170b57fa3d459b5f1ec53b6607cc728a0767fdd5"}
Sep 06 05:15:25.595068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:25.595025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" event={"ID":"3b92348c-0d65-4080-be68-00562ca27791","Type":"ContainerStarted","Data":"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"}
Sep 06 05:15:25.614745 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:25.614693    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" podStartSLOduration=2.614672113 podStartE2EDuration="2.614672113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:25.612723562 +0000 UTC m=+1187.701801255" watchObservedRunningTime="2026-09-06 05:15:25.614672113 +0000 UTC m=+1187.703749807"
Sep 06 05:15:26.288490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:26.288450    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:26.293783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:26.293756    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:26.600997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:26.600887    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:26.602172 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:26.602155    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:28.710873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.710841    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"]
Sep 06 05:15:28.714629 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.714608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:28.717132 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.717108    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:15:28.721547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.721300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"]
Sep 06 05:15:28.809318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.809275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-241a3769-9nlhz\" (UID: \"ff566709-43ec-453b-8064-d4126e6b2491\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:28.910226 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.910187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-241a3769-9nlhz\" (UID: \"ff566709-43ec-453b-8064-d4126e6b2491\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:28.910836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:28.910816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-241a3769-9nlhz\" (UID: \"ff566709-43ec-453b-8064-d4126e6b2491\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:29.027793 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:29.027698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:29.299758 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:29.299731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"]
Sep 06 05:15:29.301839 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:29.301803    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff566709_43ec_453b_8064_d4126e6b2491.slice/crio-e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2 WatchSource:0}: Error finding container e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2: Status 404 returned error can't find the container with id e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2
Sep 06 05:15:29.620472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:29.620430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz" event={"ID":"ff566709-43ec-453b-8064-d4126e6b2491","Type":"ContainerStarted","Data":"1a489ab93e4c7b2d8add12ca6e1b3cffbbcf679c67c3a68a28f5c625e4cd2215"}
Sep 06 05:15:29.620681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:29.620517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz" event={"ID":"ff566709-43ec-453b-8064-d4126e6b2491","Type":"ContainerStarted","Data":"e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2"}
Sep 06 05:15:29.634459 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:29.634407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz" podStartSLOduration=1.6343895000000002 podStartE2EDuration="1.6343895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:29.634024193 +0000 UTC m=+1191.723101887" watchObservedRunningTime="2026-09-06 05:15:29.6343895 +0000 UTC m=+1191.723467193"
Sep 06 05:15:30.627688 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:30.627656    2576 generic.go:356] "Generic (PLEG): container finished" podID="ff566709-43ec-453b-8064-d4126e6b2491" containerID="1a489ab93e4c7b2d8add12ca6e1b3cffbbcf679c67c3a68a28f5c625e4cd2215" exitCode=0
Sep 06 05:15:30.628122 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:30.627748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz" event={"ID":"ff566709-43ec-453b-8064-d4126e6b2491","Type":"ContainerDied","Data":"1a489ab93e4c7b2d8add12ca6e1b3cffbbcf679c67c3a68a28f5c625e4cd2215"}
Sep 06 05:15:31.770131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:31.770104    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:31.838547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:31.838511    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca\") pod \"ff566709-43ec-453b-8064-d4126e6b2491\" (UID: \"ff566709-43ec-453b-8064-d4126e6b2491\") "
Sep 06 05:15:31.838854 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:31.838833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff566709-43ec-453b-8064-d4126e6b2491" (UID: "ff566709-43ec-453b-8064-d4126e6b2491"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:31.939622 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:31.939581    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff566709-43ec-453b-8064-d4126e6b2491-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:32.640091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:32.640054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz" event={"ID":"ff566709-43ec-453b-8064-d4126e6b2491","Type":"ContainerDied","Data":"e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2"}
Sep 06 05:15:32.640091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:32.640075    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"
Sep 06 05:15:32.640091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:32.640100    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2cbb30c30475d69012ace108799527b2632183b657f907eda307d0b287e82d2"
Sep 06 05:15:38.409638 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:38.409599    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:15:38.411260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:38.411239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:15:38.417489 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:38.417469    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:15:38.419260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:38.419245    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:15:39.144805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.144771    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:15:39.145084 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.145043    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" podUID="bad57e2e-992b-4a15-a187-a2dfe2133d19" containerName="payload-processing" containerID="cri-o://bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d" gracePeriod=30
Sep 06 05:15:39.152417 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.152384    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:15:39.152630 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.152604    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" podUID="46820a40-76ad-4299-aea4-c641cbeed911" containerName="payload-pre-processing" containerID="cri-o://385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423" gracePeriod=30
Sep 06 05:15:39.283788 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.283752    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"]
Sep 06 05:15:39.286783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.286754    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-241a3769-9nlhz"]
Sep 06 05:15:39.409170 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.409104    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:15:39.455564 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.455539    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:15:39.520091 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520062    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qp5wh\" (UniqueName: \"kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh\") pod \"46820a40-76ad-4299-aea4-c641cbeed911\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") "
Sep 06 05:15:39.520259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520115    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44p22\" (UniqueName: \"kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22\") pod \"bad57e2e-992b-4a15-a187-a2dfe2133d19\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") "
Sep 06 05:15:39.520259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520169    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume\") pod \"46820a40-76ad-4299-aea4-c641cbeed911\" (UID: \"46820a40-76ad-4299-aea4-c641cbeed911\") "
Sep 06 05:15:39.520259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume\") pod \"bad57e2e-992b-4a15-a187-a2dfe2133d19\" (UID: \"bad57e2e-992b-4a15-a187-a2dfe2133d19\") "
Sep 06 05:15:39.520539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520504    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46820a40-76ad-4299-aea4-c641cbeed911" (UID: "46820a40-76ad-4299-aea4-c641cbeed911"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:39.520650 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.520553    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bad57e2e-992b-4a15-a187-a2dfe2133d19" (UID: "bad57e2e-992b-4a15-a187-a2dfe2133d19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:39.522453 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.522434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22" (OuterVolumeSpecName: "kube-api-access-44p22") pod "bad57e2e-992b-4a15-a187-a2dfe2133d19" (UID: "bad57e2e-992b-4a15-a187-a2dfe2133d19"). InnerVolumeSpecName "kube-api-access-44p22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:39.522510 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.522485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh" (OuterVolumeSpecName: "kube-api-access-qp5wh") pod "46820a40-76ad-4299-aea4-c641cbeed911" (UID: "46820a40-76ad-4299-aea4-c641cbeed911"). InnerVolumeSpecName "kube-api-access-qp5wh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:39.621781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.621742    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qp5wh\" (UniqueName: \"kubernetes.io/projected/46820a40-76ad-4299-aea4-c641cbeed911-kube-api-access-qp5wh\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:39.621781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.621773    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44p22\" (UniqueName: \"kubernetes.io/projected/bad57e2e-992b-4a15-a187-a2dfe2133d19-kube-api-access-44p22\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:39.621781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.621782    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46820a40-76ad-4299-aea4-c641cbeed911-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:39.621781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.621790    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad57e2e-992b-4a15-a187-a2dfe2133d19-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:39.682966 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.682936    2576 generic.go:356] "Generic (PLEG): container finished" podID="bad57e2e-992b-4a15-a187-a2dfe2133d19" containerID="bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d" exitCode=0
Sep 06 05:15:39.683162 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.683002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"
Sep 06 05:15:39.683162 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.683025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" event={"ID":"bad57e2e-992b-4a15-a187-a2dfe2133d19","Type":"ContainerDied","Data":"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"}
Sep 06 05:15:39.683162 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.683120    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h" event={"ID":"bad57e2e-992b-4a15-a187-a2dfe2133d19","Type":"ContainerDied","Data":"377fbbc245c8f59d0452c13a9b38cfaf141be7f5b1b2ac0c37e4d25169fb10c7"}
Sep 06 05:15:39.683162 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.683153    2576 scope.go:117] "RemoveContainer" containerID="bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"
Sep 06 05:15:39.684703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.684485    2576 generic.go:356] "Generic (PLEG): container finished" podID="46820a40-76ad-4299-aea4-c641cbeed911" containerID="385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423" exitCode=0
Sep 06 05:15:39.684703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.684527    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"
Sep 06 05:15:39.684703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.684586    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" event={"ID":"46820a40-76ad-4299-aea4-c641cbeed911","Type":"ContainerDied","Data":"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"}
Sep 06 05:15:39.684703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.684638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb" event={"ID":"46820a40-76ad-4299-aea4-c641cbeed911","Type":"ContainerDied","Data":"c3cbd3d79ada2e7f15f324a5148f61d602aca12afe4feacf7d5313cfe38e7cc9"}
Sep 06 05:15:39.692386 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.692364    2576 scope.go:117] "RemoveContainer" containerID="bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"
Sep 06 05:15:39.692818 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:15:39.692772    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d\": container with ID starting with bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d not found: ID does not exist" containerID="bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"
Sep 06 05:15:39.692920 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.692830    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d"} err="failed to get container status \"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d\": rpc error: code = NotFound desc = could not find container \"bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d\": container with ID starting with bd29d665dc6b234deacbc91047f63702d60f51b95858db64322750502dfa152d not found: ID does not exist"
Sep 06 05:15:39.692920 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.692857    2576 scope.go:117] "RemoveContainer" containerID="385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"
Sep 06 05:15:39.701851 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.701833    2576 scope.go:117] "RemoveContainer" containerID="385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"
Sep 06 05:15:39.702151 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:15:39.702129    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423\": container with ID starting with 385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423 not found: ID does not exist" containerID="385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"
Sep 06 05:15:39.702211 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.702158    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423"} err="failed to get container status \"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423\": rpc error: code = NotFound desc = could not find container \"385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423\": container with ID starting with 385442d40618cf1619ff07e65e711805bb8869cd0b690bde61ac1a06fd768423 not found: ID does not exist"
Sep 06 05:15:39.705597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.705577    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:15:39.709312 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.709292    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-241a3769-77bd46dfb5-mcm6h"]
Sep 06 05:15:39.719119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.719085    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:15:39.721172 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:39.721153    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-241a3769-5f555f84f4-n4fgb"]
Sep 06 05:15:40.386185 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:40.386154    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46820a40-76ad-4299-aea4-c641cbeed911" path="/var/lib/kubelet/pods/46820a40-76ad-4299-aea4-c641cbeed911/volumes"
Sep 06 05:15:40.386503 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:40.386490    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bad57e2e-992b-4a15-a187-a2dfe2133d19" path="/var/lib/kubelet/pods/bad57e2e-992b-4a15-a187-a2dfe2133d19/volumes"
Sep 06 05:15:40.386799 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:40.386786    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff566709-43ec-453b-8064-d4126e6b2491" path="/var/lib/kubelet/pods/ff566709-43ec-453b-8064-d4126e6b2491/volumes"
Sep 06 05:15:52.430709 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:52.430674    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:52.431212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:52.430956    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" podUID="3b92348c-0d65-4080-be68-00562ca27791" containerName="istio-proxy" containerID="cri-o://57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0" gracePeriod=30
Sep 06 05:15:53.842706 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.842667    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"]
Sep 06 05:15:53.843220 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843204    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff566709-43ec-453b-8064-d4126e6b2491" containerName="revoke-keys"
Sep 06 05:15:53.843276 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843224    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff566709-43ec-453b-8064-d4126e6b2491" containerName="revoke-keys"
Sep 06 05:15:53.843276 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843247    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46820a40-76ad-4299-aea4-c641cbeed911" containerName="payload-pre-processing"
Sep 06 05:15:53.843351 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843281    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46820a40-76ad-4299-aea4-c641cbeed911" containerName="payload-pre-processing"
Sep 06 05:15:53.843351 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843304    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bad57e2e-992b-4a15-a187-a2dfe2133d19" containerName="payload-processing"
Sep 06 05:15:53.843351 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843313    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bad57e2e-992b-4a15-a187-a2dfe2133d19" containerName="payload-processing"
Sep 06 05:15:53.843445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843395    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bad57e2e-992b-4a15-a187-a2dfe2133d19" containerName="payload-processing"
Sep 06 05:15:53.843445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843408    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff566709-43ec-453b-8064-d4126e6b2491" containerName="revoke-keys"
Sep 06 05:15:53.843445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.843418    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46820a40-76ad-4299-aea4-c641cbeed911" containerName="payload-pre-processing"
Sep 06 05:15:53.847813 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.847798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:53.850069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.850047    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:15:53.853735 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.853714    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"]
Sep 06 05:15:53.960880 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:53.960845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf\" (UID: \"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:54.061819 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.061784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf\" (UID: \"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:54.062390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.062370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf\" (UID: \"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:54.159400 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.159324    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:54.426664 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.426636    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"]
Sep 06 05:15:54.428711 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:15:54.428686    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f77d3a_7d2f_436f_a4b8_cacdae6f8fea.slice/crio-50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259 WatchSource:0}: Error finding container 50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259: Status 404 returned error can't find the container with id 50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259
Sep 06 05:15:54.766947 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.766913    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf" event={"ID":"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea","Type":"ContainerStarted","Data":"2668a2faf31e169b143aebcb957a7518fa1445682df9c5f35c5850040d3cd940"}
Sep 06 05:15:54.767114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.766970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf" event={"ID":"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea","Type":"ContainerStarted","Data":"50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259"}
Sep 06 05:15:54.780457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:54.780404    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf" podStartSLOduration=1.780386467 podStartE2EDuration="1.780386467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:15:54.779646429 +0000 UTC m=+1216.868724123" watchObservedRunningTime="2026-09-06 05:15:54.780386467 +0000 UTC m=+1216.869464160"
Sep 06 05:15:55.779332 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:55.773864    2576 generic.go:356] "Generic (PLEG): container finished" podID="d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" containerID="2668a2faf31e169b143aebcb957a7518fa1445682df9c5f35c5850040d3cd940" exitCode=0
Sep 06 05:15:55.779332 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:55.774127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf" event={"ID":"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea","Type":"ContainerDied","Data":"2668a2faf31e169b143aebcb957a7518fa1445682df9c5f35c5850040d3cd940"}
Sep 06 05:15:56.603600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:56.603559    2576 patch_prober.go:28] interesting pod/e2e-noten-c53189-openshift-default-75565d77f8-g4shq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:15:56.603779 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:56.603630    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" podUID="3b92348c-0d65-4080-be68-00562ca27791" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:15:56.910397 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:56.910372    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:56.990133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:56.990102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca\") pod \"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea\" (UID: \"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea\") "
Sep 06 05:15:56.990412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:56.990389    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" (UID: "d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:57.090871 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.090837    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.683971 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.683944    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:57.696501 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696417    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696501 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696461    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696501 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6pcx\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696589    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696712    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.696753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696751    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.697048 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:57.697048 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.696789    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token\") pod \"3b92348c-0d65-4080-be68-00562ca27791\" (UID: \"3b92348c-0d65-4080-be68-00562ca27791\") "
Sep 06 05:15:57.697154 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.697127    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:15:57.697215 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.697156    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:57.697364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.697327    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:57.697481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.697380    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.697558 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.697515    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:57.699959 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.699867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:15:57.700264 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.700235    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx" (OuterVolumeSpecName: "kube-api-access-j6pcx") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "kube-api-access-j6pcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:57.700638 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.700614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:15:57.701181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.701154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b92348c-0d65-4080-be68-00562ca27791" (UID: "3b92348c-0d65-4080-be68-00562ca27791"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:15:57.786401 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.786371    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"
Sep 06 05:15:57.786583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.786367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf" event={"ID":"d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea","Type":"ContainerDied","Data":"50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259"}
Sep 06 05:15:57.786583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.786527    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50dd2c36d0dceace58b95388fd5538a278a048df185cde6094ffbf9f9d54d259"
Sep 06 05:15:57.787856 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.787832    2576 generic.go:356] "Generic (PLEG): container finished" podID="3b92348c-0d65-4080-be68-00562ca27791" containerID="57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0" exitCode=0
Sep 06 05:15:57.788007 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.787875    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"
Sep 06 05:15:57.788007 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.787938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" event={"ID":"3b92348c-0d65-4080-be68-00562ca27791","Type":"ContainerDied","Data":"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"}
Sep 06 05:15:57.788007 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.787992    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq" event={"ID":"3b92348c-0d65-4080-be68-00562ca27791","Type":"ContainerDied","Data":"c9a2290a800fad50d08506d1170b57fa3d459b5f1ec53b6607cc728a0767fdd5"}
Sep 06 05:15:57.788114 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.788022    2576 scope.go:117] "RemoveContainer" containerID="57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"
Sep 06 05:15:57.799348 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799329    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b92348c-0d65-4080-be68-00562ca27791-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799352    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799367    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b92348c-0d65-4080-be68-00562ca27791-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799378    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799390    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6pcx\" (UniqueName: \"kubernetes.io/projected/3b92348c-0d65-4080-be68-00562ca27791-kube-api-access-j6pcx\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799402    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799415    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.799457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.799427    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b92348c-0d65-4080-be68-00562ca27791-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:15:57.800687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.800672    2576 scope.go:117] "RemoveContainer" containerID="57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"
Sep 06 05:15:57.801015 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:15:57.800992    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0\": container with ID starting with 57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0 not found: ID does not exist" containerID="57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"
Sep 06 05:15:57.801086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.801025    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0"} err="failed to get container status \"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0\": rpc error: code = NotFound desc = could not find container \"57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0\": container with ID starting with 57019e5b41faacf6c14da479bf5aaf1b74b3b2b4702f18f1603680c337a2f6e0 not found: ID does not exist"
Sep 06 05:15:57.812193 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.812162    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:57.814302 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:57.814283    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-c53189-openshift-default-75565d77f8-g4shq"]
Sep 06 05:15:58.386205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:15:58.386176    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b92348c-0d65-4080-be68-00562ca27791" path="/var/lib/kubelet/pods/3b92348c-0d65-4080-be68-00562ca27791/volumes"
Sep 06 05:16:03.910174 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910142    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:16:03.910577 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910553    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" containerName="revoke-keys"
Sep 06 05:16:03.910577 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" containerName="revoke-keys"
Sep 06 05:16:03.910656 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910588    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b92348c-0d65-4080-be68-00562ca27791" containerName="istio-proxy"
Sep 06 05:16:03.910656 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910594    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b92348c-0d65-4080-be68-00562ca27791" containerName="istio-proxy"
Sep 06 05:16:03.910723 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910660    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b92348c-0d65-4080-be68-00562ca27791" containerName="istio-proxy"
Sep 06 05:16:03.910723 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.910671    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" containerName="revoke-keys"
Sep 06 05:16:03.915402 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.915381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:03.918125 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.918106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-e24961\""
Sep 06 05:16:03.918329 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.918310    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-e24961-dockercfg-qwks4\""
Sep 06 05:16:03.928576 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.928553    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:16:03.950369 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.950334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:16:03.953784 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.953761    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:03.955583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.955558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs5cp\" (UniqueName: \"kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:03.955667 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.955618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:03.967124 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:03.967100    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:16:04.056376 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.056340    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.056579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.056409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs5cp\" (UniqueName: \"kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.056579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.056505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.056579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.056533    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g4pf\" (UniqueName: \"kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.057539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.057512    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.066388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.066357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs5cp\" (UniqueName: \"kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp\") pod \"payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.129937 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.129887    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:16:04.130177 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.130152    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" podUID="2a3d364b-a587-4a9e-a247-c65dd68564b9" containerName="payload-processing" containerID="cri-o://e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95" gracePeriod=30
Sep 06 05:16:04.134468 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.134441    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:16:04.134881 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.134845    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" podUID="19a82f33-a549-46e8-8272-6b82f4ef1b82" containerName="payload-pre-processing" containerID="cri-o://94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62" gracePeriod=30
Sep 06 05:16:04.160571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.160149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g4pf\" (UniqueName: \"kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.160571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.160253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.161255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.161232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.171249 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.171219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g4pf\" (UniqueName: \"kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf\") pod \"payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.225457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.225431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.264169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.264129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.432059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.432034    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:16:04.444380 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.444358    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:16:04.462766 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.462736    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume\") pod \"2a3d364b-a587-4a9e-a247-c65dd68564b9\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") "
Sep 06 05:16:04.462940 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.462888    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b76sx\" (UniqueName: \"kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx\") pod \"2a3d364b-a587-4a9e-a247-c65dd68564b9\" (UID: \"2a3d364b-a587-4a9e-a247-c65dd68564b9\") "
Sep 06 05:16:04.463157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.463136    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a3d364b-a587-4a9e-a247-c65dd68564b9" (UID: "2a3d364b-a587-4a9e-a247-c65dd68564b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:04.463247 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.463215    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a3d364b-a587-4a9e-a247-c65dd68564b9-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.465189 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.465161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx" (OuterVolumeSpecName: "kube-api-access-b76sx") pod "2a3d364b-a587-4a9e-a247-c65dd68564b9" (UID: "2a3d364b-a587-4a9e-a247-c65dd68564b9"). InnerVolumeSpecName "kube-api-access-b76sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:04.504877 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.504813    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"]
Sep 06 05:16:04.508140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.508111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:16:04.511611 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:04.511582    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28024cc_5cf7_42ca_86f8_89d91e30de14.slice/crio-bf85a4ec3a958fa70569417e05c5808f4eafdec70140e92b31e4266003e662bf WatchSource:0}: Error finding container bf85a4ec3a958fa70569417e05c5808f4eafdec70140e92b31e4266003e662bf: Status 404 returned error can't find the container with id bf85a4ec3a958fa70569417e05c5808f4eafdec70140e92b31e4266003e662bf
Sep 06 05:16:04.512159 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.512133    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d3a1cb-vbpsf"]
Sep 06 05:16:04.562370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.562348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:16:04.563982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.563966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume\") pod \"19a82f33-a549-46e8-8272-6b82f4ef1b82\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") "
Sep 06 05:16:04.564060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.564001    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v45rv\" (UniqueName: \"kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv\") pod \"19a82f33-a549-46e8-8272-6b82f4ef1b82\" (UID: \"19a82f33-a549-46e8-8272-6b82f4ef1b82\") "
Sep 06 05:16:04.564307 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.564289    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b76sx\" (UniqueName: \"kubernetes.io/projected/2a3d364b-a587-4a9e-a247-c65dd68564b9-kube-api-access-b76sx\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.564361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.564344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19a82f33-a549-46e8-8272-6b82f4ef1b82" (UID: "19a82f33-a549-46e8-8272-6b82f4ef1b82"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:04.566027 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:04.566001    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf364c76_e22e_43ba_a3d2_b2a15a5ddc6f.slice/crio-f95f93946ebeeeb64b9389f1dd2ea4cf259ad85bfc2d509bc10b273f1ba601ba WatchSource:0}: Error finding container f95f93946ebeeeb64b9389f1dd2ea4cf259ad85bfc2d509bc10b273f1ba601ba: Status 404 returned error can't find the container with id f95f93946ebeeeb64b9389f1dd2ea4cf259ad85bfc2d509bc10b273f1ba601ba
Sep 06 05:16:04.566369 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.566344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv" (OuterVolumeSpecName: "kube-api-access-v45rv") pod "19a82f33-a549-46e8-8272-6b82f4ef1b82" (UID: "19a82f33-a549-46e8-8272-6b82f4ef1b82"). InnerVolumeSpecName "kube-api-access-v45rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:04.665481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.665452    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a82f33-a549-46e8-8272-6b82f4ef1b82-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.665481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.665479    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v45rv\" (UniqueName: \"kubernetes.io/projected/19a82f33-a549-46e8-8272-6b82f4ef1b82-kube-api-access-v45rv\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:04.834777 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.834685    2576 generic.go:356] "Generic (PLEG): container finished" podID="19a82f33-a549-46e8-8272-6b82f4ef1b82" containerID="94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62" exitCode=0
Sep 06 05:16:04.834777 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.834738    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"
Sep 06 05:16:04.835054 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.834771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" event={"ID":"19a82f33-a549-46e8-8272-6b82f4ef1b82","Type":"ContainerDied","Data":"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"}
Sep 06 05:16:04.835054 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.834852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2" event={"ID":"19a82f33-a549-46e8-8272-6b82f4ef1b82","Type":"ContainerDied","Data":"57caebf9d4be4e71da611bcdb4ff1e9e98a44a56df4b95a556c6e9462bd3da4a"}
Sep 06 05:16:04.835054 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.834876    2576 scope.go:117] "RemoveContainer" containerID="94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"
Sep 06 05:16:04.836320 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.836297    2576 generic.go:356] "Generic (PLEG): container finished" podID="2a3d364b-a587-4a9e-a247-c65dd68564b9" containerID="e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95" exitCode=0
Sep 06 05:16:04.836411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.836331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" event={"ID":"2a3d364b-a587-4a9e-a247-c65dd68564b9","Type":"ContainerDied","Data":"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"}
Sep 06 05:16:04.836411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.836368    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"
Sep 06 05:16:04.836411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.836389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp" event={"ID":"2a3d364b-a587-4a9e-a247-c65dd68564b9","Type":"ContainerDied","Data":"c2d651a6b8b156ad1fb9756dddac6a024b97819a5ba10f1500938bc9c428afb5"}
Sep 06 05:16:04.838413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.838384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" event={"ID":"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f","Type":"ContainerStarted","Data":"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"}
Sep 06 05:16:04.838542 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.838434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" event={"ID":"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f","Type":"ContainerStarted","Data":"f95f93946ebeeeb64b9389f1dd2ea4cf259ad85bfc2d509bc10b273f1ba601ba"}
Sep 06 05:16:04.838632 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.838549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:04.840712 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.840686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" event={"ID":"f28024cc-5cf7-42ca-86f8-89d91e30de14","Type":"ContainerStarted","Data":"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"}
Sep 06 05:16:04.840806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.840734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" event={"ID":"f28024cc-5cf7-42ca-86f8-89d91e30de14","Type":"ContainerStarted","Data":"bf85a4ec3a958fa70569417e05c5808f4eafdec70140e92b31e4266003e662bf"}
Sep 06 05:16:04.840806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.840783    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:04.845839 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.845815    2576 scope.go:117] "RemoveContainer" containerID="94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"
Sep 06 05:16:04.846271 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:16:04.846251    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62\": container with ID starting with 94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62 not found: ID does not exist" containerID="94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"
Sep 06 05:16:04.846363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.846284    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62"} err="failed to get container status \"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62\": rpc error: code = NotFound desc = could not find container \"94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62\": container with ID starting with 94b563f7628430e4dd1a789160e5439d0d46923c03b5c618489664e8eff91f62 not found: ID does not exist"
Sep 06 05:16:04.846363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.846310    2576 scope.go:117] "RemoveContainer" containerID="e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"
Sep 06 05:16:04.855195 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.855145    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" podStartSLOduration=1.855127878 podStartE2EDuration="1.855127878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:04.852589652 +0000 UTC m=+1226.941667361" watchObservedRunningTime="2026-09-06 05:16:04.855127878 +0000 UTC m=+1226.944205571"
Sep 06 05:16:04.856021 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.855998    2576 scope.go:117] "RemoveContainer" containerID="e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"
Sep 06 05:16:04.856382 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:16:04.856362    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95\": container with ID starting with e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95 not found: ID does not exist" containerID="e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"
Sep 06 05:16:04.856446 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.856390    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95"} err="failed to get container status \"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95\": rpc error: code = NotFound desc = could not find container \"e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95\": container with ID starting with e8501d0beec5e3e80f1d37118c16ec78fdc1655e585772aaf5b6a42095c0cc95 not found: ID does not exist"
Sep 06 05:16:04.869731 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.869699    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:16:04.871486 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.871459    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-d3a1cb-6559fdb6c8-6dhq2"]
Sep 06 05:16:04.883996 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.883940    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" podStartSLOduration=1.883890075 podStartE2EDuration="1.883890075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:04.88317159 +0000 UTC m=+1226.972249286" watchObservedRunningTime="2026-09-06 05:16:04.883890075 +0000 UTC m=+1226.972967769"
Sep 06 05:16:04.898654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.898612    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:16:04.900707 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:04.900680    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-d3a1cb-5d75846f89-cjhzp"]
Sep 06 05:16:06.389683 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:06.389649    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a82f33-a549-46e8-8272-6b82f4ef1b82" path="/var/lib/kubelet/pods/19a82f33-a549-46e8-8272-6b82f4ef1b82/volumes"
Sep 06 05:16:06.390167 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:06.390151    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a3d364b-a587-4a9e-a247-c65dd68564b9" path="/var/lib/kubelet/pods/2a3d364b-a587-4a9e-a247-c65dd68564b9/volumes"
Sep 06 05:16:06.390553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:06.390540    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea" path="/var/lib/kubelet/pods/d5f77d3a-7d2f-436f-a4b8-cacdae6f8fea/volumes"
Sep 06 05:16:11.097436 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097403    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"]
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097764    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a3d364b-a587-4a9e-a247-c65dd68564b9" containerName="payload-processing"
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097775    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a3d364b-a587-4a9e-a247-c65dd68564b9" containerName="payload-processing"
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097792    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a82f33-a549-46e8-8272-6b82f4ef1b82" containerName="payload-pre-processing"
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097798    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a82f33-a549-46e8-8272-6b82f4ef1b82" containerName="payload-pre-processing"
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097852    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a3d364b-a587-4a9e-a247-c65dd68564b9" containerName="payload-processing"
Sep 06 05:16:11.097872 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.097865    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a82f33-a549-46e8-8272-6b82f4ef1b82" containerName="payload-pre-processing"
Sep 06 05:16:11.102147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.102130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:11.105410 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.105383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:16:11.107165 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.107142    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"]
Sep 06 05:16:11.229027 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.228986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7\" (UID: \"e3ace537-b198-4d14-8ff7-a3ac2fb17367\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:11.330430 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.330392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7\" (UID: \"e3ace537-b198-4d14-8ff7-a3ac2fb17367\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:11.331305 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.331274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7\" (UID: \"e3ace537-b198-4d14-8ff7-a3ac2fb17367\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:11.414676 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.414580    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:11.693840 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.693817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"]
Sep 06 05:16:11.695889 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:11.695860    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ace537_b198_4d14_8ff7_a3ac2fb17367.slice/crio-e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4 WatchSource:0}: Error finding container e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4: Status 404 returned error can't find the container with id e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4
Sep 06 05:16:11.881142 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:11.881109    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7" event={"ID":"e3ace537-b198-4d14-8ff7-a3ac2fb17367","Type":"ContainerStarted","Data":"e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4"}
Sep 06 05:16:12.887187 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:12.887150    2576 generic.go:356] "Generic (PLEG): container finished" podID="e3ace537-b198-4d14-8ff7-a3ac2fb17367" containerID="22eff85b8d7bbd26002c7fdbe75bfc2ff3c4b0c2c18a067b6002e6f7b84efd9d" exitCode=0
Sep 06 05:16:12.887574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:12.887233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7" event={"ID":"e3ace537-b198-4d14-8ff7-a3ac2fb17367","Type":"ContainerDied","Data":"22eff85b8d7bbd26002c7fdbe75bfc2ff3c4b0c2c18a067b6002e6f7b84efd9d"}
Sep 06 05:16:14.028690 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.028663    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:14.153507 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.153404    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca\") pod \"e3ace537-b198-4d14-8ff7-a3ac2fb17367\" (UID: \"e3ace537-b198-4d14-8ff7-a3ac2fb17367\") "
Sep 06 05:16:14.153781 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.153757    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3ace537-b198-4d14-8ff7-a3ac2fb17367" (UID: "e3ace537-b198-4d14-8ff7-a3ac2fb17367"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:14.254512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.254472    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ace537-b198-4d14-8ff7-a3ac2fb17367-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:14.903538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.903508    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"
Sep 06 05:16:14.903538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.903520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7" event={"ID":"e3ace537-b198-4d14-8ff7-a3ac2fb17367","Type":"ContainerDied","Data":"e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4"}
Sep 06 05:16:14.903756 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:14.903586    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e574a3afa28829851ecdd9fe9f8f2c7297db451f0ff0d74eebd968be61a47ce4"
Sep 06 05:16:15.851041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:15.851010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:16:15.851473 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:15.851088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:16:21.710375 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.710337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"]
Sep 06 05:16:21.712225 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.712194    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-9035bc-29811195-wmhxg"]
Sep 06 05:16:21.757450 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.757418    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:16:21.757737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.757712    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" podUID="b78d79b3-d067-47aa-a856-51559edfae69" containerName="payload-processing" containerID="cri-o://2b751aba1b49cf470fb73ff22d3653f6b8720f31fd789cc9fe5fd3116e44e0f4" gracePeriod=30
Sep 06 05:16:21.762582 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.762556    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:16:21.763292 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.763239    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" podUID="6784308e-74bb-4e83-b284-9b9c01d88382" containerName="payload-pre-processing" containerID="cri-o://eea215d92d8f245ab8742f63a0701ce76f83a5245a2a4a6cb62a77d514bdf5c0" gracePeriod=30
Sep 06 05:16:21.913862 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.913814    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"]
Sep 06 05:16:21.915643 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.915612    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9035bc-kt5r7"]
Sep 06 05:16:21.945455 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.945418    2576 generic.go:356] "Generic (PLEG): container finished" podID="6784308e-74bb-4e83-b284-9b9c01d88382" containerID="eea215d92d8f245ab8742f63a0701ce76f83a5245a2a4a6cb62a77d514bdf5c0" exitCode=0
Sep 06 05:16:21.945640 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.945486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" event={"ID":"6784308e-74bb-4e83-b284-9b9c01d88382","Type":"ContainerDied","Data":"eea215d92d8f245ab8742f63a0701ce76f83a5245a2a4a6cb62a77d514bdf5c0"}
Sep 06 05:16:21.947028 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.947002    2576 generic.go:356] "Generic (PLEG): container finished" podID="b78d79b3-d067-47aa-a856-51559edfae69" containerID="2b751aba1b49cf470fb73ff22d3653f6b8720f31fd789cc9fe5fd3116e44e0f4" exitCode=0
Sep 06 05:16:21.947153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:21.947063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" event={"ID":"b78d79b3-d067-47aa-a856-51559edfae69","Type":"ContainerDied","Data":"2b751aba1b49cf470fb73ff22d3653f6b8720f31fd789cc9fe5fd3116e44e0f4"}
Sep 06 05:16:22.044639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.044616    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:16:22.047466 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.047446    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:16:22.120863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.120828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkpg9\" (UniqueName: \"kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9\") pod \"b78d79b3-d067-47aa-a856-51559edfae69\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") "
Sep 06 05:16:22.121032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.120890    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume\") pod \"6784308e-74bb-4e83-b284-9b9c01d88382\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") "
Sep 06 05:16:22.121032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.120952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlp4r\" (UniqueName: \"kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r\") pod \"6784308e-74bb-4e83-b284-9b9c01d88382\" (UID: \"6784308e-74bb-4e83-b284-9b9c01d88382\") "
Sep 06 05:16:22.121032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.121016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume\") pod \"b78d79b3-d067-47aa-a856-51559edfae69\" (UID: \"b78d79b3-d067-47aa-a856-51559edfae69\") "
Sep 06 05:16:22.121404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.121377    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6784308e-74bb-4e83-b284-9b9c01d88382" (UID: "6784308e-74bb-4e83-b284-9b9c01d88382"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:22.121404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.121393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b78d79b3-d067-47aa-a856-51559edfae69" (UID: "b78d79b3-d067-47aa-a856-51559edfae69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:16:22.123279 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.123255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9" (OuterVolumeSpecName: "kube-api-access-kkpg9") pod "b78d79b3-d067-47aa-a856-51559edfae69" (UID: "b78d79b3-d067-47aa-a856-51559edfae69"). InnerVolumeSpecName "kube-api-access-kkpg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:22.123384 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.123256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r" (OuterVolumeSpecName: "kube-api-access-zlp4r") pod "6784308e-74bb-4e83-b284-9b9c01d88382" (UID: "6784308e-74bb-4e83-b284-9b9c01d88382"). InnerVolumeSpecName "kube-api-access-zlp4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:16:22.222624 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.222537    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b78d79b3-d067-47aa-a856-51559edfae69-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:22.222624 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.222563    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkpg9\" (UniqueName: \"kubernetes.io/projected/b78d79b3-d067-47aa-a856-51559edfae69-kube-api-access-kkpg9\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:22.222624 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.222574    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6784308e-74bb-4e83-b284-9b9c01d88382-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:22.222624 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.222583    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlp4r\" (UniqueName: \"kubernetes.io/projected/6784308e-74bb-4e83-b284-9b9c01d88382-kube-api-access-zlp4r\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:16:22.385721 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.385687    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c6ba1ea-127f-4a69-a833-b27796052ded" path="/var/lib/kubelet/pods/1c6ba1ea-127f-4a69-a833-b27796052ded/volumes"
Sep 06 05:16:22.386022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.386009    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3ace537-b198-4d14-8ff7-a3ac2fb17367" path="/var/lib/kubelet/pods/e3ace537-b198-4d14-8ff7-a3ac2fb17367/volumes"
Sep 06 05:16:22.953862 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.953831    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"
Sep 06 05:16:22.954389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.953877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj" event={"ID":"6784308e-74bb-4e83-b284-9b9c01d88382","Type":"ContainerDied","Data":"8acc67673e811ba670e5fc16837d983d0992c62e5ddfbf4a3f7660e8c9cb26f1"}
Sep 06 05:16:22.954389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.953973    2576 scope.go:117] "RemoveContainer" containerID="eea215d92d8f245ab8742f63a0701ce76f83a5245a2a4a6cb62a77d514bdf5c0"
Sep 06 05:16:22.955428 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.955402    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"
Sep 06 05:16:22.955514 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.955451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7" event={"ID":"b78d79b3-d067-47aa-a856-51559edfae69","Type":"ContainerDied","Data":"c0357611af9b874dcd29a1ce8af8b355e8fae969a83701df717df78236ccc111"}
Sep 06 05:16:22.963184 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.963152    2576 scope.go:117] "RemoveContainer" containerID="2b751aba1b49cf470fb73ff22d3653f6b8720f31fd789cc9fe5fd3116e44e0f4"
Sep 06 05:16:22.972893 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.972875    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:16:22.976480 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.976458    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-9035bc-6dfd96c86b-zbmjj"]
Sep 06 05:16:22.987093 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.987071    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:16:22.992633 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:22.992608    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-9035bc-645695948c-stzr7"]
Sep 06 05:16:24.385954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:24.385896    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6784308e-74bb-4e83-b284-9b9c01d88382" path="/var/lib/kubelet/pods/6784308e-74bb-4e83-b284-9b9c01d88382/volumes"
Sep 06 05:16:24.386457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:24.386411    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b78d79b3-d067-47aa-a856-51559edfae69" path="/var/lib/kubelet/pods/b78d79b3-d067-47aa-a856-51559edfae69/volumes"
Sep 06 05:16:27.648832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.648798    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:16:27.649373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649350    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b78d79b3-d067-47aa-a856-51559edfae69" containerName="payload-processing"
Sep 06 05:16:27.649456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649376    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b78d79b3-d067-47aa-a856-51559edfae69" containerName="payload-processing"
Sep 06 05:16:27.649456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649403    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6784308e-74bb-4e83-b284-9b9c01d88382" containerName="payload-pre-processing"
Sep 06 05:16:27.649456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649411    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6784308e-74bb-4e83-b284-9b9c01d88382" containerName="payload-pre-processing"
Sep 06 05:16:27.649456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649421    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3ace537-b198-4d14-8ff7-a3ac2fb17367" containerName="revoke-keys"
Sep 06 05:16:27.649456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649430    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3ace537-b198-4d14-8ff7-a3ac2fb17367" containerName="revoke-keys"
Sep 06 05:16:27.649702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649516    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b78d79b3-d067-47aa-a856-51559edfae69" containerName="payload-processing"
Sep 06 05:16:27.649702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649530    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3ace537-b198-4d14-8ff7-a3ac2fb17367" containerName="revoke-keys"
Sep 06 05:16:27.649702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.649540    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6784308e-74bb-4e83-b284-9b9c01d88382" containerName="payload-pre-processing"
Sep 06 05:16:27.654579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.654558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.657300 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.657257    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-401f17-openshift-default-dockercfg-g2xdv\""
Sep 06 05:16:27.664517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.664490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:16:27.774349 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774349 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774507    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmhvn\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.774778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.774625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.875762 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.875762 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.875996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmhvn\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876223    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876495 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876617 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.876617 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.876603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.878251 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.878227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.878668 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.878652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.883568 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.883544    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.883780 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.883765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmhvn\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn\") pod \"e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") " pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:27.967201 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:27.967169    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:28.243605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.243528    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:16:28.246321 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:28.246295    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969f1593_db41_4205_bc6f_fc6689d0f881.slice/crio-654fbcb17a034b25ab420f9567cddcd74534f2476443e3b0a0315f4fa33cff81 WatchSource:0}: Error finding container 654fbcb17a034b25ab420f9567cddcd74534f2476443e3b0a0315f4fa33cff81: Status 404 returned error can't find the container with id 654fbcb17a034b25ab420f9567cddcd74534f2476443e3b0a0315f4fa33cff81
Sep 06 05:16:28.248625 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.248592    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:28.248713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.248661    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:28.248713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.248692    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:28.992218 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.992182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerStarted","Data":"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"}
Sep 06 05:16:28.992666 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:28.992239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerStarted","Data":"654fbcb17a034b25ab420f9567cddcd74534f2476443e3b0a0315f4fa33cff81"}
Sep 06 05:16:29.012270 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:29.012207    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podStartSLOduration=2.012192542 podStartE2EDuration="2.012192542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:29.009771015 +0000 UTC m=+1251.098848709" watchObservedRunningTime="2026-09-06 05:16:29.012192542 +0000 UTC m=+1251.101270236"
Sep 06 05:16:29.967839 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:29.967793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:29.973198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:29.973172    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:29.997184 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:29.997154    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:29.998072 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:29.998056    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:32.681765 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.681733    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:16:32.685436 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.685416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.687946 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.687923    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-401f17-dockercfg-xg5sp\""
Sep 06 05:16:32.688094 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.687957    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-401f17\""
Sep 06 05:16:32.692856 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.692831    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:16:32.829406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.829367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9v5k\" (UniqueName: \"kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.829406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.829406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.930561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.930527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9v5k\" (UniqueName: \"kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.930561 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.930566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.931388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.931365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.941074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.941009    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9v5k\" (UniqueName: \"kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k\") pod \"payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:32.998582 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:32.998559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:33.271848 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:33.271820    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:16:33.273678 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:33.273653    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00935404_27ab_4dc1_8bf5_b992465e25a8.slice/crio-8217584477e79c54b819dcfd61976f37514d4bb0bf881f914e1bc151e8f7653d WatchSource:0}: Error finding container 8217584477e79c54b819dcfd61976f37514d4bb0bf881f914e1bc151e8f7653d: Status 404 returned error can't find the container with id 8217584477e79c54b819dcfd61976f37514d4bb0bf881f914e1bc151e8f7653d
Sep 06 05:16:34.020289 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.020255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" event={"ID":"00935404-27ab-4dc1-8bf5-b992465e25a8","Type":"ContainerStarted","Data":"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"}
Sep 06 05:16:34.020661 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.020315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" event={"ID":"00935404-27ab-4dc1-8bf5-b992465e25a8","Type":"ContainerStarted","Data":"8217584477e79c54b819dcfd61976f37514d4bb0bf881f914e1bc151e8f7653d"}
Sep 06 05:16:34.020661 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.020337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:34.056673 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.056625    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" podStartSLOduration=2.056612106 podStartE2EDuration="2.056612106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:34.05507038 +0000 UTC m=+1256.144148073" watchObservedRunningTime="2026-09-06 05:16:34.056612106 +0000 UTC m=+1256.145689799"
Sep 06 05:16:34.318454 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.318374    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:16:34.321736 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.321719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.324115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.324095    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-494740\""
Sep 06 05:16:34.324227 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.324120    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-494740-dockercfg-45jm7\""
Sep 06 05:16:34.331342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.331317    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:16:34.444890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.444858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.445074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.444929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v8gv\" (UniqueName: \"kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.545535 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.545483    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v8gv\" (UniqueName: \"kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.545727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.545703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.546441 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.546419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.555247 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.555222    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v8gv\" (UniqueName: \"kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv\") pod \"payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.633278 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.633198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:34.910800 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:34.910775    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:16:34.912793 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:34.912768    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c998e34_8012_48c2_b10e_f2e7ca5f693a.slice/crio-c9fdf80ad84010c38235d1e39b55088e086ab6afc87b82e993dd5a1ef52f64c0 WatchSource:0}: Error finding container c9fdf80ad84010c38235d1e39b55088e086ab6afc87b82e993dd5a1ef52f64c0: Status 404 returned error can't find the container with id c9fdf80ad84010c38235d1e39b55088e086ab6afc87b82e993dd5a1ef52f64c0
Sep 06 05:16:35.032626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:35.032588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" event={"ID":"1c998e34-8012-48c2-b10e-f2e7ca5f693a","Type":"ContainerStarted","Data":"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"}
Sep 06 05:16:35.033041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:35.032756    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:35.033041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:35.032794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" event={"ID":"1c998e34-8012-48c2-b10e-f2e7ca5f693a","Type":"ContainerStarted","Data":"c9fdf80ad84010c38235d1e39b55088e086ab6afc87b82e993dd5a1ef52f64c0"}
Sep 06 05:16:35.047963 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:35.047915    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" podStartSLOduration=1.047890497 podStartE2EDuration="1.047890497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:35.046278795 +0000 UTC m=+1257.135356490" watchObservedRunningTime="2026-09-06 05:16:35.047890497 +0000 UTC m=+1257.136968189"
Sep 06 05:16:45.035383 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:45.035348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:16:46.039084 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:46.039041    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 05:16:46.039464 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:46.039097    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:16:46.042836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:46.042814    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:16:48.128735 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.128697    2576 generic.go:356] "Generic (PLEG): container finished" podID="969f1593-db41-4205-bc6f-fc6689d0f881" containerID="9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c" exitCode=0
Sep 06 05:16:48.129190 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.128751    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerDied","Data":"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"}
Sep 06 05:16:48.129234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.129196    2576 scope.go:117] "RemoveContainer" containerID="9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"
Sep 06 05:16:48.130330 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.130295    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:48.130424 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.130409    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:48.130462 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:48.130446    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:49.137133 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:49.137091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerStarted","Data":"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"}
Sep 06 05:16:49.967991 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:49.967951    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:49.968323 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:49.968198    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 05:16:49.968521 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:49.968495    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 05:16:50.968440 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:50.968405    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 05:16:50.969060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:50.969026    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 05:16:51.968210 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:51.968168    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 05:16:51.968431 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:51.968242    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 05:16:52.967779 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:52.967742    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 05:16:52.968317 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:52.967814    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 05:16:53.967672 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:53.967635    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 06 05:16:53.967974 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:53.967690    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 06 05:16:54.971425 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:54.971397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:54.971800 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:54.971676    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:54.972469 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:54.972452    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:16:57.856192 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.856158    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:16:57.859203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.859180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.861627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.861608    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-167b5b-openshift-default-dockercfg-l9ptm\""
Sep 06 05:16:57.871731 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.871708    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:16:57.980993 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.980957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981168 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981168 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981168 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981154    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981358 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981358 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981439 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:57.981497 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:57.981479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd2vl\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.082846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.082813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.082846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.082856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.082944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.082975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd2vl\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083474 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083474 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083474 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.083945 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.083926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.085826 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.085798    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.085963 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.085860    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.091411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.091386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.091547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.091517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd2vl\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl\") pod \"e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") " pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.171763 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.171734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:16:58.454766 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.454732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:16:58.456071 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:16:58.456043    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb1d907_f2f2_4613_8e8a_bfc05ed36455.slice/crio-58c8d2534039c93c952ff77b1ebaf4b84e93f3ddd49f2c35f52bbc6216e90408 WatchSource:0}: Error finding container 58c8d2534039c93c952ff77b1ebaf4b84e93f3ddd49f2c35f52bbc6216e90408: Status 404 returned error can't find the container with id 58c8d2534039c93c952ff77b1ebaf4b84e93f3ddd49f2c35f52bbc6216e90408
Sep 06 05:16:58.458326 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.458290    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:58.458432 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.458372    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:58.458432 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:58.458417    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:16:59.192980 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:59.192944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerStarted","Data":"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"}
Sep 06 05:16:59.193477 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:59.192993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerStarted","Data":"58c8d2534039c93c952ff77b1ebaf4b84e93f3ddd49f2c35f52bbc6216e90408"}
Sep 06 05:16:59.213336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:16:59.213279    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podStartSLOduration=2.213262439 podStartE2EDuration="2.213262439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:16:59.211793363 +0000 UTC m=+1281.300871058" watchObservedRunningTime="2026-09-06 05:16:59.213262439 +0000 UTC m=+1281.302340132"
Sep 06 05:17:00.172603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:00.172560    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:00.178991 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:00.178965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:00.198879 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:00.198850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:00.199695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:00.199678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:02.781293 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.781261    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:17:02.784188 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.784173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:02.787837 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.787814    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-167b5b\""
Sep 06 05:17:02.787997 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.787935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-167b5b-dockercfg-5d7c9\""
Sep 06 05:17:02.797332 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.797311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:17:02.941588 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.941550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mss2b\" (UniqueName: \"kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:02.941952 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:02.941927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.042493 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.042413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mss2b\" (UniqueName: \"kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.042493 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.042466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.043111 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.043091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.051017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.050989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mss2b\" (UniqueName: \"kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b\") pod \"payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.094493 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.094468    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:03.361889 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:03.361865    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:17:03.363515 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:17:03.363489    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd5b011_21a7_43a7_98c6_0caccf573d7d.slice/crio-6aaa64032a1699ffb10537d5741aac749aec16b37a35f3e47deb654b94bc72c0 WatchSource:0}: Error finding container 6aaa64032a1699ffb10537d5741aac749aec16b37a35f3e47deb654b94bc72c0: Status 404 returned error can't find the container with id 6aaa64032a1699ffb10537d5741aac749aec16b37a35f3e47deb654b94bc72c0
Sep 06 05:17:04.222463 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:04.222430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" event={"ID":"0cd5b011-21a7-43a7-98c6-0caccf573d7d","Type":"ContainerStarted","Data":"2d45d71dca63428fb2924924569593e8ebb5fecbf91f71d3094ae191384f35d6"}
Sep 06 05:17:04.222849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:04.222479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" event={"ID":"0cd5b011-21a7-43a7-98c6-0caccf573d7d","Type":"ContainerStarted","Data":"6aaa64032a1699ffb10537d5741aac749aec16b37a35f3e47deb654b94bc72c0"}
Sep 06 05:17:04.222849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:04.222573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:04.238074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:04.238025    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" podStartSLOduration=2.238009933 podStartE2EDuration="2.238009933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:17:04.236026215 +0000 UTC m=+1286.325103921" watchObservedRunningTime="2026-09-06 05:17:04.238009933 +0000 UTC m=+1286.327087625"
Sep 06 05:17:09.778683 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.778640    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:17:09.786757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.786733    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.797954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.797462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:17:09.798679 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.798655    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1b19002b-openshift-default-dockercfg-67mvp\""
Sep 06 05:17:09.910956 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.910920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911149 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.910991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911149 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911149 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911149 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wdpv\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911223    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911283    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:09.911392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:09.911332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.011977 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.011940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.011985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wdpv\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012489 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012410    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012489 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012713 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.012853 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.012832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.014725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.014702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.014966 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.014950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.019280 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.019257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.019695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.019674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wdpv\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv\") pod \"e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") " pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.115527 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.115427    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:10.415958 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.415935    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:17:10.418165 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:17:10.418138    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096c2f35_e513_415a_9eac_c9a7df984c80.slice/crio-1785dcba6130672781b42b84f7525c86b798b2e74235c4e8fd5f6adc45bd9a4a WatchSource:0}: Error finding container 1785dcba6130672781b42b84f7525c86b798b2e74235c4e8fd5f6adc45bd9a4a: Status 404 returned error can't find the container with id 1785dcba6130672781b42b84f7525c86b798b2e74235c4e8fd5f6adc45bd9a4a
Sep 06 05:17:10.420879 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.420847    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:10.420967 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.420933    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:10.421013 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:10.420964    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:11.260355 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:11.260322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerStarted","Data":"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"}
Sep 06 05:17:11.260767 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:11.260383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerStarted","Data":"1785dcba6130672781b42b84f7525c86b798b2e74235c4e8fd5f6adc45bd9a4a"}
Sep 06 05:17:11.280956 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:11.280872    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podStartSLOduration=2.280848604 podStartE2EDuration="2.280848604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:17:11.27734465 +0000 UTC m=+1293.366422345" watchObservedRunningTime="2026-09-06 05:17:11.280848604 +0000 UTC m=+1293.369926297"
Sep 06 05:17:12.115538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:12.115509    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:12.121467 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:12.121442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:12.266034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:12.266008    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:12.267034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:12.267015    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:15.230103 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:15.230068    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:17:16.240089 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:16.240046    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 06 05:17:16.240490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:16.240119    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 06 05:17:18.309884 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.309850    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerID="999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107" exitCode=0
Sep 06 05:17:18.310337 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.309932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerDied","Data":"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"}
Sep 06 05:17:18.310380 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.310364    2576 scope.go:117] "RemoveContainer" containerID="999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"
Sep 06 05:17:18.311681 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.311655    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:18.311742 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.311727    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:18.311782 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:18.311761    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:19.317433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:19.317397    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerStarted","Data":"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"}
Sep 06 05:17:20.172476 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:20.172427    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:20.172833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:20.172803    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 06 05:17:20.173075 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:20.173045    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 06 05:17:21.172339 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:21.172298    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 06 05:17:21.172965 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:21.172930    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 06 05:17:22.173198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:22.173163    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 06 05:17:22.173787 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:22.173758    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 06 05:17:23.172428 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:23.172380    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 06 05:17:23.172653 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:23.172452    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 06 05:17:24.172720 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:24.172682    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 06 05:17:24.173338 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:24.173304    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 06 05:17:25.183296 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:25.183254    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:25.184526 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:25.184495    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:25.189770 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:25.189753    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:17:32.405516 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.405479    2576 generic.go:356] "Generic (PLEG): container finished" podID="096c2f35-e513-415a-9eac-c9a7df984c80" containerID="821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe" exitCode=0
Sep 06 05:17:32.406041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.405553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerDied","Data":"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"}
Sep 06 05:17:32.406041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.406018    2576 scope.go:117] "RemoveContainer" containerID="821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"
Sep 06 05:17:32.407348 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.407310    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:32.407441 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.407404    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:32.407487 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:32.407447    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:17:33.412674 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:33.412642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerStarted","Data":"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"}
Sep 06 05:17:34.115975 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:34.115936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:34.116182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:34.116158    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:34.116254 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:34.116229    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:35.115730 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:35.115697    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:35.116262 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:35.115777    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:36.116136 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:36.116104    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:36.116538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:36.116159    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:37.116660 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:37.116623    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:37.117087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:37.116685    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:38.116384 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:38.116348    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:38.116615 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:38.116413    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:38.511365 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:38.511331    2576 scope.go:117] "RemoveContainer" containerID="ec5fd66f05abf4011681b9d6e0f4ec686161b20eec16d2857ccf8f75f19fab5f"
Sep 06 05:17:39.116104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:39.116069    2576 patch_prober.go:28] interesting pod/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 06 05:17:39.116297 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:39.116169    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 06 05:17:40.120884 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:40.120853    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:40.121363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:40.121281    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:17:40.122049 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:17:40.122032    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:18:33.099964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.099919    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"]
Sep 06 05:18:33.102769 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.102749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:33.105287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.105264    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:18:33.111207 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.111178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"]
Sep 06 05:18:33.149968 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.149922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc\" (UID: \"b837aaa5-88f5-4918-8141-02472f6ed1b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:33.250630 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.250590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc\" (UID: \"b837aaa5-88f5-4918-8141-02472f6ed1b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:33.251385 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.251351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc\" (UID: \"b837aaa5-88f5-4918-8141-02472f6ed1b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:33.415702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.415613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:33.685464 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.685441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"]
Sep 06 05:18:33.687238 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:18:33.687209    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb837aaa5_88f5_4918_8141_02472f6ed1b6.slice/crio-75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c WatchSource:0}: Error finding container 75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c: Status 404 returned error can't find the container with id 75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c
Sep 06 05:18:33.748313 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:33.748283    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc" event={"ID":"b837aaa5-88f5-4918-8141-02472f6ed1b6","Type":"ContainerStarted","Data":"75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c"}
Sep 06 05:18:34.755863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:34.755826    2576 generic.go:356] "Generic (PLEG): container finished" podID="b837aaa5-88f5-4918-8141-02472f6ed1b6" containerID="b0904e2220ee73717c479a3771907daff57a612032ea2d0d5595c9c89dfa14a3" exitCode=0
Sep 06 05:18:34.756315 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:34.755925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc" event={"ID":"b837aaa5-88f5-4918-8141-02472f6ed1b6","Type":"ContainerDied","Data":"b0904e2220ee73717c479a3771907daff57a612032ea2d0d5595c9c89dfa14a3"}
Sep 06 05:18:35.897066 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:35.897042    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:35.977635 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:35.977596    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca\") pod \"b837aaa5-88f5-4918-8141-02472f6ed1b6\" (UID: \"b837aaa5-88f5-4918-8141-02472f6ed1b6\") "
Sep 06 05:18:35.978002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:35.977981    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b837aaa5-88f5-4918-8141-02472f6ed1b6" (UID: "b837aaa5-88f5-4918-8141-02472f6ed1b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:18:36.078401 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:36.078311    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b837aaa5-88f5-4918-8141-02472f6ed1b6-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:18:36.767982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:36.767950    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"
Sep 06 05:18:36.767982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:36.767974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc" event={"ID":"b837aaa5-88f5-4918-8141-02472f6ed1b6","Type":"ContainerDied","Data":"75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c"}
Sep 06 05:18:36.768206 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:36.768027    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75f5523aca8604fba8ea54dcf6de6d2793dd6b96f4ba2807a162e7fb97e0a64c"
Sep 06 05:18:38.531988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:38.531954    2576 scope.go:117] "RemoveContainer" containerID="c5c6e55acbd23e83b1a0ec604039631749dbd9467935a2b0b6855a7d49449f85"
Sep 06 05:18:38.541333 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:38.541306    2576 scope.go:117] "RemoveContainer" containerID="3b5bfff6f50bde1adf9854de1d46a4077c11f3a17db4966b467a3257e62e5dd4"
Sep 06 05:18:48.850648 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:48.850611    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"]
Sep 06 05:18:48.853603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:48.853571    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1b19002b-hb2jc"]
Sep 06 05:18:50.386518 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:18:50.386478    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b837aaa5-88f5-4918-8141-02472f6ed1b6" path="/var/lib/kubelet/pods/b837aaa5-88f5-4918-8141-02472f6ed1b6/volumes"
Sep 06 05:19:38.566204 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:19:38.566118    2576 scope.go:117] "RemoveContainer" containerID="cb6db4fa92d27fc72337e4bf3951b9a91ab97ee9aec130ed3c487c9200b86f4b"
Sep 06 05:19:38.574818 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:19:38.574800    2576 scope.go:117] "RemoveContainer" containerID="0fa595ae233039ee914e0b2060a98176e4af0ffd69802ec36d6dcb2e35ad4bcb"
Sep 06 05:19:38.582478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:19:38.582458    2576 scope.go:117] "RemoveContainer" containerID="fd9c6e111f1d030f9603209d2ea33d067f708aea467f09eb75466c673cb4e4f4"
Sep 06 05:20:01.868894 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:01.868856    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:20:01.869301 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:01.869108    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy" containerID="cri-o://eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7" gracePeriod=30
Sep 06 05:20:07.134539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.134511    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:20:07.279890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.279854    2576 generic.go:356] "Generic (PLEG): container finished" podID="096c2f35-e513-415a-9eac-c9a7df984c80" containerID="eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7" exitCode=0
Sep 06 05:20:07.280095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.279938    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"
Sep 06 05:20:07.280095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.279950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerDied","Data":"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"}
Sep 06 05:20:07.280095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.280001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn" event={"ID":"096c2f35-e513-415a-9eac-c9a7df984c80","Type":"ContainerDied","Data":"1785dcba6130672781b42b84f7525c86b798b2e74235c4e8fd5f6adc45bd9a4a"}
Sep 06 05:20:07.280095 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.280023    2576 scope.go:117] "RemoveContainer" containerID="eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"
Sep 06 05:20:07.289043 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.289024    2576 scope.go:117] "RemoveContainer" containerID="821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"
Sep 06 05:20:07.297218 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.297202    2576 scope.go:117] "RemoveContainer" containerID="eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"
Sep 06 05:20:07.297483 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:20:07.297460    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7\": container with ID starting with eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7 not found: ID does not exist" containerID="eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"
Sep 06 05:20:07.297531 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.297496    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7"} err="failed to get container status \"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7\": rpc error: code = NotFound desc = could not find container \"eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7\": container with ID starting with eb64da3be575a7d4e1102b5ed151921067348e0ed809210b668c35576ae6f9f7 not found: ID does not exist"
Sep 06 05:20:07.297531 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.297514    2576 scope.go:117] "RemoveContainer" containerID="821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"
Sep 06 05:20:07.297775 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:20:07.297757    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe\": container with ID starting with 821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe not found: ID does not exist" containerID="821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"
Sep 06 05:20:07.297820 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.297781    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe"} err="failed to get container status \"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe\": rpc error: code = NotFound desc = could not find container \"821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe\": container with ID starting with 821d277e9ec01a9ed929adf38c4f1d0daaa519ad7d75523bbd1fd480a7715fbe not found: ID does not exist"
Sep 06 05:20:07.305103 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305084    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wdpv\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305115    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305176 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305259 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305236    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305350 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305367    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305409    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305528 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:07.305528 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305429    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:07.305528 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305511    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket\") pod \"096c2f35-e513-415a-9eac-c9a7df984c80\" (UID: \"096c2f35-e513-415a-9eac-c9a7df984c80\") "
Sep 06 05:20:07.305697 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data" (OuterVolumeSpecName: "istio-data") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:07.305756 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305724    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:07.305809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:20:07.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305934    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096c2f35-e513-415a-9eac-c9a7df984c80-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305953    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305963    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305972    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.305983 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.305984    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.307617 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.307582    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:20:07.307725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.307634    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token" (OuterVolumeSpecName: "istio-token") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:20:07.307833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.307818    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv" (OuterVolumeSpecName: "kube-api-access-8wdpv") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "kube-api-access-8wdpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:20:07.307954 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.307939    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "096c2f35-e513-415a-9eac-c9a7df984c80" (UID: "096c2f35-e513-415a-9eac-c9a7df984c80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:20:07.407273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.407231    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.407273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.407265    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096c2f35-e513-415a-9eac-c9a7df984c80-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.407273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.407277    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wdpv\" (UniqueName: \"kubernetes.io/projected/096c2f35-e513-415a-9eac-c9a7df984c80-kube-api-access-8wdpv\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.407529 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.407289    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096c2f35-e513-415a-9eac-c9a7df984c80-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:20:07.601821 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.601792    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:20:07.605499 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:07.605473    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1b19002b-openshift-default-6849b4df5b-t6bpn"]
Sep 06 05:20:08.385620 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:08.385580    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" path="/var/lib/kubelet/pods/096c2f35-e513-415a-9eac-c9a7df984c80/volumes"
Sep 06 05:20:38.440600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:38.440572    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:20:38.444657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:38.444638    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:20:38.447948 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:38.447927    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:20:38.452173 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:38.452157    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:20:55.807860 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.807827    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808281    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b837aaa5-88f5-4918-8141-02472f6ed1b6" containerName="revoke-keys"
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808295    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b837aaa5-88f5-4918-8141-02472f6ed1b6" containerName="revoke-keys"
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808312    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808328    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808334    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808387    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808399    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="096c2f35-e513-415a-9eac-c9a7df984c80" containerName="istio-proxy"
Sep 06 05:20:55.808571 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.808409    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b837aaa5-88f5-4918-8141-02472f6ed1b6" containerName="revoke-keys"
Sep 06 05:20:55.812998 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.812970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.815438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.815417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-71403e-openshift-default-dockercfg-gp4pr\""
Sep 06 05:20:55.822882 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.822855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:20:55.961832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.961794    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.961852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.961878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmzc9\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.961948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.962011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962058 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.962038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.962076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.962107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:55.962263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:55.962143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063178 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063178 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063135    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063178 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmzc9\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063727 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.063892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.063821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.064092 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.064069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.064332 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.064307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.065815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.065788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.066027 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.066011    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.071036 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.071012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.071138 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.071115    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmzc9\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9\") pod \"e2e-api-a-71403e-openshift-default-78cd564684-f54p7\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") " pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.126069 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.126026    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:56.402319 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.402291    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:20:56.404098 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:20:56.404071    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70c4d47_7ffb_4482_bf0b_7a45ad70c854.slice/crio-dba4c08d39b0a3503cf02bb077c06adace541534dbea364f1cda873594f82a30 WatchSource:0}: Error finding container dba4c08d39b0a3503cf02bb077c06adace541534dbea364f1cda873594f82a30: Status 404 returned error can't find the container with id dba4c08d39b0a3503cf02bb077c06adace541534dbea364f1cda873594f82a30
Sep 06 05:20:56.406768 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.406736    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:20:56.406822 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.406807    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:20:56.406855 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.406837    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:20:56.547846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.547801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerStarted","Data":"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"}
Sep 06 05:20:56.548059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.547859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerStarted","Data":"dba4c08d39b0a3503cf02bb077c06adace541534dbea364f1cda873594f82a30"}
Sep 06 05:20:56.569548 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:56.569487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podStartSLOduration=1.56946211 podStartE2EDuration="1.56946211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:20:56.566042658 +0000 UTC m=+1518.655120356" watchObservedRunningTime="2026-09-06 05:20:56.56946211 +0000 UTC m=+1518.658539805"
Sep 06 05:20:57.126546 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:57.126514    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:57.128014 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:57.127991    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:20:57.128132 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:57.128040    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:20:58.130656 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:58.130621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:58.557783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:58.557753    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:20:58.558770 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:20:58.558750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:21:00.197325 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.197245    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:21:00.201132 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.201114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.203629 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.203608    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-71403e-dockercfg-dbj6v\""
Sep 06 05:21:00.203753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.203656    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-71403e\""
Sep 06 05:21:00.211089 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.211065    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:21:00.305142 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.305033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwps6\" (UniqueName: \"kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.305448 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.305419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.406915 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.406864    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.407112 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.407039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwps6\" (UniqueName: \"kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.407579 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.407559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.414597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.414573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwps6\" (UniqueName: \"kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6\") pod \"payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.512874 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.512783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:00.786892 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:00.786868    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:21:00.788851 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:21:00.788819    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b0495b9_8b75_4982_8b3d_541f263bc279.slice/crio-e7a14a71c83b13a24ed1287a05decb183430ce05d9a878da736913383e3f0010 WatchSource:0}: Error finding container e7a14a71c83b13a24ed1287a05decb183430ce05d9a878da736913383e3f0010: Status 404 returned error can't find the container with id e7a14a71c83b13a24ed1287a05decb183430ce05d9a878da736913383e3f0010
Sep 06 05:21:01.576723 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:01.576690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" event={"ID":"6b0495b9-8b75-4982-8b3d-541f263bc279","Type":"ContainerStarted","Data":"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"}
Sep 06 05:21:01.577141 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:01.576756    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" event={"ID":"6b0495b9-8b75-4982-8b3d-541f263bc279","Type":"ContainerStarted","Data":"e7a14a71c83b13a24ed1287a05decb183430ce05d9a878da736913383e3f0010"}
Sep 06 05:21:01.577141 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:01.576787    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:01.592190 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:01.592144    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" podStartSLOduration=1.592128023 podStartE2EDuration="1.592128023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:01.590280938 +0000 UTC m=+1523.679358643" watchObservedRunningTime="2026-09-06 05:21:01.592128023 +0000 UTC m=+1523.681205716"
Sep 06 05:21:12.586200 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:12.585466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:21:19.672828 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.672790    2576 generic.go:356] "Generic (PLEG): container finished" podID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerID="6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91" exitCode=0
Sep 06 05:21:19.673304 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.672869    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerDied","Data":"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"}
Sep 06 05:21:19.673353 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.673329    2576 scope.go:117] "RemoveContainer" containerID="6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"
Sep 06 05:21:19.674623 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.674597    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:19.674691 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.674662    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:19.674733 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:19.674697    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:20.684273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:20.684231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerStarted","Data":"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"}
Sep 06 05:21:21.127177 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:21.127087    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:21:21.127329 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:21.127301    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:21:21.127373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:21.127342    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:21:22.126985 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:22.126946    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:21:22.127412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:22.127013    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:21:23.127000 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:23.126961    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:21:23.127501 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:23.127022    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:21:24.127205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.127171    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:21:24.127610 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.127225    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:21:24.217611 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.217566    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:21:24.220379 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.220353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.222840 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.222819    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-596168-openshift-default-dockercfg-x76j5\""
Sep 06 05:21:24.233583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.233562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:21:24.349918 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.349875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350100 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.349958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350100 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.349991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350100 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d68k\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350100 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350283 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350283 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350283 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350210    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.350283 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.350259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451386 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d68k\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.451943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.452180 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.452180 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.451977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.452305 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.452183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.452305 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.452270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.453859 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.453836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.454115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.454099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.459342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.459322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.459439 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.459327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d68k\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k\") pod \"e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") " pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.534288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.534257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:24.829986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.829961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:21:24.831759 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:21:24.831733    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bd0409_26bb_47dd_8648_914393a30b61.slice/crio-9644e78b5551bef56ccc3ca8921e8d286f69259cad64a7ba08ad75b5608ae4df WatchSource:0}: Error finding container 9644e78b5551bef56ccc3ca8921e8d286f69259cad64a7ba08ad75b5608ae4df: Status 404 returned error can't find the container with id 9644e78b5551bef56ccc3ca8921e8d286f69259cad64a7ba08ad75b5608ae4df
Sep 06 05:21:24.834205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.834151    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:24.834286 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.834242    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:24.834286 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:24.834270    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:25.126747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:25.126656    2576 patch_prober.go:28] interesting pod/e2e-api-a-71403e-openshift-default-78cd564684-f54p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 06 05:21:25.126747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:25.126711    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 06 05:21:25.720889 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:25.720841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerStarted","Data":"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"}
Sep 06 05:21:25.721394 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:25.720927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerStarted","Data":"9644e78b5551bef56ccc3ca8921e8d286f69259cad64a7ba08ad75b5608ae4df"}
Sep 06 05:21:25.742158 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:25.742110    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podStartSLOduration=1.742094392 podStartE2EDuration="1.742094392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:25.739994583 +0000 UTC m=+1547.829072277" watchObservedRunningTime="2026-09-06 05:21:25.742094392 +0000 UTC m=+1547.831172101"
Sep 06 05:21:26.131199 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.131114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:21:26.131449 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.131432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:21:26.132107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.132086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:21:26.534833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.534796    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:26.540539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.540515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:26.726760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.726725    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:26.727820 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:26.727799    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:38.621119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:38.621087    2576 scope.go:117] "RemoveContainer" containerID="7759239492cfdba724401f05e4732b5eba4ea194ea7086c0ba360dc9d2c6ee81"
Sep 06 05:21:38.629741 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:38.629721    2576 scope.go:117] "RemoveContainer" containerID="1a489ab93e4c7b2d8add12ca6e1b3cffbbcf679c67c3a68a28f5c625e4cd2215"
Sep 06 05:21:38.637453 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:38.637439    2576 scope.go:117] "RemoveContainer" containerID="2d48ee21410c5f4515425d267c6e6791dd2a966b2c20f9c536c5c74551028865"
Sep 06 05:21:38.644778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:38.644766    2576 scope.go:117] "RemoveContainer" containerID="254afba5fa69c922b6914546618f28bdafc16569f6952e6e2cf293e334215b3f"
Sep 06 05:21:51.859364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.859330    2576 generic.go:356] "Generic (PLEG): container finished" podID="33bd0409-26bb-47dd-8648-914393a30b61" containerID="82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c" exitCode=0
Sep 06 05:21:51.859795 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.859403    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerDied","Data":"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"}
Sep 06 05:21:51.859858 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.859844    2576 scope.go:117] "RemoveContainer" containerID="82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"
Sep 06 05:21:51.861143 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.861115    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:51.861202 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.861187    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:51.861242 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:51.861226    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:21:52.866666 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:52.866635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerStarted","Data":"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"}
Sep 06 05:21:53.534815 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:53.534775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:53.535847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:53.535820    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 05:21:53.536109 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:53.536071    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 05:21:54.535562 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:54.535528    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 05:21:54.536019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:54.535594    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 05:21:55.504827 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.504791    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"]
Sep 06 05:21:55.507423 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.507408    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:55.509784 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.509758    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:21:55.513822 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.513793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"]
Sep 06 05:21:55.535392 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.535361    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 05:21:55.535517 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.535416    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 05:21:55.591922 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.590878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-71403e-pc8vf\" (UID: \"f33f373a-d182-4552-a3bf-e8e24326bec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:55.692012 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.691967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-71403e-pc8vf\" (UID: \"f33f373a-d182-4552-a3bf-e8e24326bec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:55.692962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.692933    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-71403e-pc8vf\" (UID: \"f33f373a-d182-4552-a3bf-e8e24326bec8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:55.820480 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:55.820381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:56.109019 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.108994    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"]
Sep 06 05:21:56.110786 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:21:56.110761    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33f373a_d182_4552_a3bf_e8e24326bec8.slice/crio-58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463 WatchSource:0}: Error finding container 58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463: Status 404 returned error can't find the container with id 58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463
Sep 06 05:21:56.534861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.534831    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 05:21:56.535011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.534881    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 05:21:56.726719 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.726681    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:56.916109 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.916021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf" event={"ID":"f33f373a-d182-4552-a3bf-e8e24326bec8","Type":"ContainerStarted","Data":"9ffb2003b7d33455fd3e762b73a6bb68a038ffeb17a538b69fb5d3103877dc18"}
Sep 06 05:21:56.916109 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.916066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf" event={"ID":"f33f373a-d182-4552-a3bf-e8e24326bec8","Type":"ContainerStarted","Data":"58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463"}
Sep 06 05:21:56.930728 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:56.930401    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf" podStartSLOduration=1.930383086 podStartE2EDuration="1.930383086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:21:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:21:56.93037794 +0000 UTC m=+1579.019455634" watchObservedRunningTime="2026-09-06 05:21:56.930383086 +0000 UTC m=+1579.019460781"
Sep 06 05:21:57.535106 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:57.535071    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Sep 06 05:21:57.535306 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:57.535132    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Sep 06 05:21:57.923061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:57.923026    2576 generic.go:356] "Generic (PLEG): container finished" podID="f33f373a-d182-4552-a3bf-e8e24326bec8" containerID="9ffb2003b7d33455fd3e762b73a6bb68a038ffeb17a538b69fb5d3103877dc18" exitCode=0
Sep 06 05:21:57.923442 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:57.923107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf" event={"ID":"f33f373a-d182-4552-a3bf-e8e24326bec8","Type":"ContainerDied","Data":"9ffb2003b7d33455fd3e762b73a6bb68a038ffeb17a538b69fb5d3103877dc18"}
Sep 06 05:21:58.539514 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:58.539484    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:58.540570 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:58.540547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:21:59.064318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.064296    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:21:59.226582 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.226496    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca\") pod \"f33f373a-d182-4552-a3bf-e8e24326bec8\" (UID: \"f33f373a-d182-4552-a3bf-e8e24326bec8\") "
Sep 06 05:21:59.226884 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.226859    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f33f373a-d182-4552-a3bf-e8e24326bec8" (UID: "f33f373a-d182-4552-a3bf-e8e24326bec8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:21:59.328211 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.328183    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f33f373a-d182-4552-a3bf-e8e24326bec8-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:21:59.935438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.935406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf" event={"ID":"f33f373a-d182-4552-a3bf-e8e24326bec8","Type":"ContainerDied","Data":"58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463"}
Sep 06 05:21:59.935612 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.935457    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58c5e78d0d0a5bd18bbbad5ae839d47e4ff1b2b6b91c4d746945ec2c06340463"
Sep 06 05:21:59.935612 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:21:59.935415    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"
Sep 06 05:22:10.976800 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:10.976761    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:22:10.977349 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:10.977041    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" podUID="6b0495b9-8b75-4982-8b3d-541f263bc279" containerName="payload-pre-processing" containerID="cri-o://80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6" gracePeriod=30
Sep 06 05:22:11.115358 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.115326    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"]
Sep 06 05:22:11.117734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.117695    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-71403e-pc8vf"]
Sep 06 05:22:11.230242 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.230188    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:22:11.252233 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.252203    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume\") pod \"6b0495b9-8b75-4982-8b3d-541f263bc279\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") "
Sep 06 05:22:11.252391 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.252285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwps6\" (UniqueName: \"kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6\") pod \"6b0495b9-8b75-4982-8b3d-541f263bc279\" (UID: \"6b0495b9-8b75-4982-8b3d-541f263bc279\") "
Sep 06 05:22:11.252660 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.252631    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b0495b9-8b75-4982-8b3d-541f263bc279" (UID: "6b0495b9-8b75-4982-8b3d-541f263bc279"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:11.254600 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.254572    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6" (OuterVolumeSpecName: "kube-api-access-kwps6") pod "6b0495b9-8b75-4982-8b3d-541f263bc279" (UID: "6b0495b9-8b75-4982-8b3d-541f263bc279"). InnerVolumeSpecName "kube-api-access-kwps6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:11.354187 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.354152    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwps6\" (UniqueName: \"kubernetes.io/projected/6b0495b9-8b75-4982-8b3d-541f263bc279-kube-api-access-kwps6\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:11.354187 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.354182    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b0495b9-8b75-4982-8b3d-541f263bc279-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:11.999753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.999710    2576 generic.go:356] "Generic (PLEG): container finished" podID="6b0495b9-8b75-4982-8b3d-541f263bc279" containerID="80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6" exitCode=0
Sep 06 05:22:12.000299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.999765    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"
Sep 06 05:22:12.000299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.999768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" event={"ID":"6b0495b9-8b75-4982-8b3d-541f263bc279","Type":"ContainerDied","Data":"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"}
Sep 06 05:22:12.000299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.999882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr" event={"ID":"6b0495b9-8b75-4982-8b3d-541f263bc279","Type":"ContainerDied","Data":"e7a14a71c83b13a24ed1287a05decb183430ce05d9a878da736913383e3f0010"}
Sep 06 05:22:12.000299 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:11.999937    2576 scope.go:117] "RemoveContainer" containerID="80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"
Sep 06 05:22:12.010307 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.010281    2576 scope.go:117] "RemoveContainer" containerID="80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"
Sep 06 05:22:12.010621 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:12.010599    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6\": container with ID starting with 80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6 not found: ID does not exist" containerID="80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"
Sep 06 05:22:12.010692 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.010632    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6"} err="failed to get container status \"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6\": rpc error: code = NotFound desc = could not find container \"80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6\": container with ID starting with 80ddac54a7dac02d6e8d1c42bbc949061a576a1e818a7ac8b1c348c045915aa6 not found: ID does not exist"
Sep 06 05:22:12.028526 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.028493    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:22:12.034610 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.034581    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-71403e-8569b55894-slrpr"]
Sep 06 05:22:12.386141 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.386063    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b0495b9-8b75-4982-8b3d-541f263bc279" path="/var/lib/kubelet/pods/6b0495b9-8b75-4982-8b3d-541f263bc279/volumes"
Sep 06 05:22:12.386419 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:12.386407    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33f373a-d182-4552-a3bf-e8e24326bec8" path="/var/lib/kubelet/pods/f33f373a-d182-4552-a3bf-e8e24326bec8/volumes"
Sep 06 05:22:17.325265 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.325228    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:22:17.325761 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.325574    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy" containerID="cri-o://049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736" gracePeriod=30
Sep 06 05:22:17.708146 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708111    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"]
Sep 06 05:22:17.708524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708511    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b0495b9-8b75-4982-8b3d-541f263bc279" containerName="payload-pre-processing"
Sep 06 05:22:17.708595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708526    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b0495b9-8b75-4982-8b3d-541f263bc279" containerName="payload-pre-processing"
Sep 06 05:22:17.708595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708538    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33f373a-d182-4552-a3bf-e8e24326bec8" containerName="revoke-keys"
Sep 06 05:22:17.708595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708543    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33f373a-d182-4552-a3bf-e8e24326bec8" containerName="revoke-keys"
Sep 06 05:22:17.708721 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708607    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b0495b9-8b75-4982-8b3d-541f263bc279" containerName="payload-pre-processing"
Sep 06 05:22:17.708721 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.708618    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33f373a-d182-4552-a3bf-e8e24326bec8" containerName="revoke-keys"
Sep 06 05:22:17.711603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.711587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:17.713967 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.713945    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:22:17.717734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.717715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"]
Sep 06 05:22:17.823459 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.823422    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-596168-5vzp4\" (UID: \"c083d7d5-b9b6-4f69-b173-8428c8479a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:17.924427 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.924382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-596168-5vzp4\" (UID: \"c083d7d5-b9b6-4f69-b173-8428c8479a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:17.925065 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:17.925045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-596168-5vzp4\" (UID: \"c083d7d5-b9b6-4f69-b173-8428c8479a4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:18.024080 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:18.023996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:18.303746 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:18.303667    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"]
Sep 06 05:22:19.041552 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:19.041519    2576 generic.go:356] "Generic (PLEG): container finished" podID="c083d7d5-b9b6-4f69-b173-8428c8479a4b" containerID="752361d92089a2f35e73af88d23123f30adc7999094b39e904115b9d1a9b16b4" exitCode=0
Sep 06 05:22:19.041875 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:19.041599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4" event={"ID":"c083d7d5-b9b6-4f69-b173-8428c8479a4b","Type":"ContainerDied","Data":"752361d92089a2f35e73af88d23123f30adc7999094b39e904115b9d1a9b16b4"}
Sep 06 05:22:19.041875 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:19.041649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4" event={"ID":"c083d7d5-b9b6-4f69-b173-8428c8479a4b","Type":"ContainerStarted","Data":"2b1c5c2da371d60ede1d0fbff39a6f26c2fcbdfa81a52498e572fc1118f0c2e9"}
Sep 06 05:22:20.186429 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:20.186401    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:20.247075 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:20.247048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca\") pod \"c083d7d5-b9b6-4f69-b173-8428c8479a4b\" (UID: \"c083d7d5-b9b6-4f69-b173-8428c8479a4b\") "
Sep 06 05:22:20.247390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:20.247366    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c083d7d5-b9b6-4f69-b173-8428c8479a4b" (UID: "c083d7d5-b9b6-4f69-b173-8428c8479a4b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:20.348283 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:20.348204    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c083d7d5-b9b6-4f69-b173-8428c8479a4b-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:21.056295 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:21.056267    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"
Sep 06 05:22:21.056295 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:21.056291    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4" event={"ID":"c083d7d5-b9b6-4f69-b173-8428c8479a4b","Type":"ContainerDied","Data":"2b1c5c2da371d60ede1d0fbff39a6f26c2fcbdfa81a52498e572fc1118f0c2e9"}
Sep 06 05:22:21.056505 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:21.056339    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b1c5c2da371d60ede1d0fbff39a6f26c2fcbdfa81a52498e572fc1118f0c2e9"
Sep 06 05:22:22.588064 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.588007    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:22:22.671051 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671023    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmzc9\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671066    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671086    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671105    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671277    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671316    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket\") pod \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\" (UID: \"e70c4d47-7ffb-4482-bf0b-7a45ad70c854\") "
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671415    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671448    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data" (OuterVolumeSpecName: "istio-data") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:22.671559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:22.671847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671618    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:22.671847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671650    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:22.671847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671820    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.671847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671838    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.672034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671852    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.672034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671865    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.672034 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.671879    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.673719 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.673686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9" (OuterVolumeSpecName: "kube-api-access-jmzc9") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "kube-api-access-jmzc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:22.673837 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.673721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token" (OuterVolumeSpecName: "istio-token") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:22.673922 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.673878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:22:22.673987 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.673965    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e70c4d47-7ffb-4482-bf0b-7a45ad70c854" (UID: "e70c4d47-7ffb-4482-bf0b-7a45ad70c854"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:22.773147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.773116    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmzc9\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-kube-api-access-jmzc9\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.773147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.773143    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.773147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.773153    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:22.773360 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:22.773161    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e70c4d47-7ffb-4482-bf0b-7a45ad70c854-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:23.068849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.068815    2576 generic.go:356] "Generic (PLEG): container finished" podID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerID="049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736" exitCode=0
Sep 06 05:22:23.069037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.068879    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"
Sep 06 05:22:23.069037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.068882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerDied","Data":"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"}
Sep 06 05:22:23.069037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.068948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7" event={"ID":"e70c4d47-7ffb-4482-bf0b-7a45ad70c854","Type":"ContainerDied","Data":"dba4c08d39b0a3503cf02bb077c06adace541534dbea364f1cda873594f82a30"}
Sep 06 05:22:23.069037 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.068974    2576 scope.go:117] "RemoveContainer" containerID="049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"
Sep 06 05:22:23.079057 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.079038    2576 scope.go:117] "RemoveContainer" containerID="6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"
Sep 06 05:22:23.087174 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.087156    2576 scope.go:117] "RemoveContainer" containerID="049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"
Sep 06 05:22:23.087433 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:23.087413    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736\": container with ID starting with 049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736 not found: ID does not exist" containerID="049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"
Sep 06 05:22:23.087478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.087442    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736"} err="failed to get container status \"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736\": rpc error: code = NotFound desc = could not find container \"049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736\": container with ID starting with 049ae0f035bd2086d269c68860aea29001a118fb4764b275748ef49adc991736 not found: ID does not exist"
Sep 06 05:22:23.087478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.087457    2576 scope.go:117] "RemoveContainer" containerID="6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"
Sep 06 05:22:23.087781 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:23.087752    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91\": container with ID starting with 6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91 not found: ID does not exist" containerID="6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"
Sep 06 05:22:23.088089 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.087789    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91"} err="failed to get container status \"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91\": rpc error: code = NotFound desc = could not find container \"6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91\": container with ID starting with 6fde33524be7b69a13ffa39b51b541371aeb8900c70ccaf3e2f728f713b7ad91 not found: ID does not exist"
Sep 06 05:22:23.091295 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.091274    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:22:23.094857 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:23.094834    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-71403e-openshift-default-78cd564684-f54p7"]
Sep 06 05:22:24.386039 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:24.386010    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" path="/var/lib/kubelet/pods/e70c4d47-7ffb-4482-bf0b-7a45ad70c854/volumes"
Sep 06 05:22:33.545988 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:33.545889    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"]
Sep 06 05:22:33.549734 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:33.549709    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-596168-5vzp4"]
Sep 06 05:22:34.389608 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:34.389575    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c083d7d5-b9b6-4f69-b173-8428c8479a4b" path="/var/lib/kubelet/pods/c083d7d5-b9b6-4f69-b173-8428c8479a4b/volumes"
Sep 06 05:22:38.673964 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:38.673936    2576 scope.go:117] "RemoveContainer" containerID="22eff85b8d7bbd26002c7fdbe75bfc2ff3c4b0c2c18a067b6002e6f7b84efd9d"
Sep 06 05:22:38.682098 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:38.682079    2576 scope.go:117] "RemoveContainer" containerID="2668a2faf31e169b143aebcb957a7518fa1445682df9c5f35c5850040d3cd940"
Sep 06 05:22:39.761057 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:39.761020    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:22:39.761468 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:39.761299    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" containerID="cri-o://7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447" gracePeriod=30
Sep 06 05:22:40.115682 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.115609    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"]
Sep 06 05:22:40.116046 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116021    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116046 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116048    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116136 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116056    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c083d7d5-b9b6-4f69-b173-8428c8479a4b" containerName="revoke-keys"
Sep 06 05:22:40.116136 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116062    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c083d7d5-b9b6-4f69-b173-8428c8479a4b" containerName="revoke-keys"
Sep 06 05:22:40.116136 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116076    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116136 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116082    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116144    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116151    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70c4d47-7ffb-4482-bf0b-7a45ad70c854" containerName="istio-proxy"
Sep 06 05:22:40.116263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.116158    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c083d7d5-b9b6-4f69-b173-8428c8479a4b" containerName="revoke-keys"
Sep 06 05:22:40.118941 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.118895    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:40.121280 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.121258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:22:40.125994 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.125934    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"]
Sep 06 05:22:40.141464 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.141440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8\" (UID: \"ddf62b49-f92e-44dd-bcb7-42f816aadcc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:40.242451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.242407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8\" (UID: \"ddf62b49-f92e-44dd-bcb7-42f816aadcc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:40.243131 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.243104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8\" (UID: \"ddf62b49-f92e-44dd-bcb7-42f816aadcc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:40.430894 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.430864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:40.703247 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:40.703217    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"]
Sep 06 05:22:40.704823 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:22:40.704792    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf62b49_f92e_44dd_bcb7_42f816aadcc7.slice/crio-6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89 WatchSource:0}: Error finding container 6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89: Status 404 returned error can't find the container with id 6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89
Sep 06 05:22:41.166951 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:41.166892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8" event={"ID":"ddf62b49-f92e-44dd-bcb7-42f816aadcc7","Type":"ContainerStarted","Data":"4316a7112f2dbe4e24d11fc1d60d7e436dfdba17f7af4ec1eaa883060fd7b95d"}
Sep 06 05:22:41.167319 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:41.166975    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8" event={"ID":"ddf62b49-f92e-44dd-bcb7-42f816aadcc7","Type":"ContainerStarted","Data":"6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89"}
Sep 06 05:22:41.181193 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:41.181145    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8" podStartSLOduration=1.181131681 podStartE2EDuration="1.181131681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:22:41.180201817 +0000 UTC m=+1623.269279511" watchObservedRunningTime="2026-09-06 05:22:41.181131681 +0000 UTC m=+1623.270209374"
Sep 06 05:22:42.173355 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:42.173319    2576 generic.go:356] "Generic (PLEG): container finished" podID="ddf62b49-f92e-44dd-bcb7-42f816aadcc7" containerID="4316a7112f2dbe4e24d11fc1d60d7e436dfdba17f7af4ec1eaa883060fd7b95d" exitCode=0
Sep 06 05:22:42.173817 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:42.173398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8" event={"ID":"ddf62b49-f92e-44dd-bcb7-42f816aadcc7","Type":"ContainerDied","Data":"4316a7112f2dbe4e24d11fc1d60d7e436dfdba17f7af4ec1eaa883060fd7b95d"}
Sep 06 05:22:43.312973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.312946    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:43.369393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.369354    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca\") pod \"ddf62b49-f92e-44dd-bcb7-42f816aadcc7\" (UID: \"ddf62b49-f92e-44dd-bcb7-42f816aadcc7\") "
Sep 06 05:22:43.369675 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.369651    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ddf62b49-f92e-44dd-bcb7-42f816aadcc7" (UID: "ddf62b49-f92e-44dd-bcb7-42f816aadcc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:43.470799 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.470710    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf62b49-f92e-44dd-bcb7-42f816aadcc7-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:43.542585 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.542548    2576 patch_prober.go:28] interesting pod/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:22:43.542735 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:43.542617    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:22:44.185418 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:44.185389    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"
Sep 06 05:22:44.185601 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:44.185388    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8" event={"ID":"ddf62b49-f92e-44dd-bcb7-42f816aadcc7","Type":"ContainerDied","Data":"6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89"}
Sep 06 05:22:44.185601 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:44.185516    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d16ee4e1ce31029be9821842394bd79766841ac87929699146a4d137984ff89"
Sep 06 05:22:45.027617 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.027558    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:22:45.085597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085569    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085773 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085773 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085646    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d68k\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085862    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085930    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.085970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085928    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:45.085970 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.085961    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.086260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.086260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086065    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert\") pod \"33bd0409-26bb-47dd-8648-914393a30b61\" (UID: \"33bd0409-26bb-47dd-8648-914393a30b61\") "
Sep 06 05:22:45.086260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086077    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:45.086260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086143    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data" (OuterVolumeSpecName: "istio-data") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:45.086260 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:45.086714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086397    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.086714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086412    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.086714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086420    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.086714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086429    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.086714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.086462    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:45.088065 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.088042    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token" (OuterVolumeSpecName: "istio-token") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:45.088282 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.088251    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k" (OuterVolumeSpecName: "kube-api-access-5d68k") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "kube-api-access-5d68k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:45.088380 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.088254    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:22:45.088380 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.088350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "33bd0409-26bb-47dd-8648-914393a30b61" (UID: "33bd0409-26bb-47dd-8648-914393a30b61"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:22:45.187224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.187191    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bd0409-26bb-47dd-8648-914393a30b61-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.187224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.187212    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bd0409-26bb-47dd-8648-914393a30b61-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.187224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.187222    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.187224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.187231    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d68k\" (UniqueName: \"kubernetes.io/projected/33bd0409-26bb-47dd-8648-914393a30b61-kube-api-access-5d68k\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.187593 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.187239    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bd0409-26bb-47dd-8648-914393a30b61-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:45.192509 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.192483    2576 generic.go:356] "Generic (PLEG): container finished" podID="33bd0409-26bb-47dd-8648-914393a30b61" containerID="7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447" exitCode=0
Sep 06 05:22:45.192628 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.192533    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"
Sep 06 05:22:45.192628 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.192567    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerDied","Data":"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"}
Sep 06 05:22:45.192702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.192626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd" event={"ID":"33bd0409-26bb-47dd-8648-914393a30b61","Type":"ContainerDied","Data":"9644e78b5551bef56ccc3ca8921e8d286f69259cad64a7ba08ad75b5608ae4df"}
Sep 06 05:22:45.192702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.192658    2576 scope.go:117] "RemoveContainer" containerID="7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"
Sep 06 05:22:45.204111 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.204090    2576 scope.go:117] "RemoveContainer" containerID="82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"
Sep 06 05:22:45.214309 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.214291    2576 scope.go:117] "RemoveContainer" containerID="7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"
Sep 06 05:22:45.214561 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:45.214542    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447\": container with ID starting with 7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447 not found: ID does not exist" containerID="7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"
Sep 06 05:22:45.214608 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.214570    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447"} err="failed to get container status \"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447\": rpc error: code = NotFound desc = could not find container \"7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447\": container with ID starting with 7d17acee58e384860baefec94f53cf15ad3d93c3322eb2a63208da1051207447 not found: ID does not exist"
Sep 06 05:22:45.214608 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.214588    2576 scope.go:117] "RemoveContainer" containerID="82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"
Sep 06 05:22:45.214856 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:45.214842    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c\": container with ID starting with 82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c not found: ID does not exist" containerID="82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"
Sep 06 05:22:45.214925 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.214859    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c"} err="failed to get container status \"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c\": rpc error: code = NotFound desc = could not find container \"82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c\": container with ID starting with 82848fbe9617bd98807fc56e38ad493d06824226c99ed56d7652b532eecacf1c not found: ID does not exist"
Sep 06 05:22:45.215395 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.215371    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:22:45.218222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:45.218200    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-596168-openshift-default-559bdfcbb4-mqrwd"]
Sep 06 05:22:46.385413 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:46.385378    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33bd0409-26bb-47dd-8648-914393a30b61" path="/var/lib/kubelet/pods/33bd0409-26bb-47dd-8648-914393a30b61/volumes"
Sep 06 05:22:50.580612 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.580578    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:22:50.581263 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.580771    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" podUID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" containerName="payload-processing" containerID="cri-o://bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832" gracePeriod=30
Sep 06 05:22:50.586128 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.586100    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:22:50.586547 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.586487    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" podUID="f28024cc-5cf7-42ca-86f8-89d91e30de14" containerName="payload-pre-processing" containerID="cri-o://ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb" gracePeriod=30
Sep 06 05:22:50.719956 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.719919    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"]
Sep 06 05:22:50.721883 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.721856    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-e24961-449z8"]
Sep 06 05:22:50.869334 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.869311    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:22:50.872388 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.872368    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:22:50.944543 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.944514    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume\") pod \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") "
Sep 06 05:22:50.944707 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.944635    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume\") pod \"f28024cc-5cf7-42ca-86f8-89d91e30de14\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") "
Sep 06 05:22:50.944707 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.944690    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs5cp\" (UniqueName: \"kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp\") pod \"f28024cc-5cf7-42ca-86f8-89d91e30de14\" (UID: \"f28024cc-5cf7-42ca-86f8-89d91e30de14\") "
Sep 06 05:22:50.944837 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.944718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g4pf\" (UniqueName: \"kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf\") pod \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\" (UID: \"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f\") "
Sep 06 05:22:50.944972 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.944944    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" (UID: "bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:50.945080 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.945057    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f28024cc-5cf7-42ca-86f8-89d91e30de14" (UID: "f28024cc-5cf7-42ca-86f8-89d91e30de14"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:22:50.945153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.945139    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:50.945190 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.945155    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f28024cc-5cf7-42ca-86f8-89d91e30de14-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:50.947011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.946983    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp" (OuterVolumeSpecName: "kube-api-access-qs5cp") pod "f28024cc-5cf7-42ca-86f8-89d91e30de14" (UID: "f28024cc-5cf7-42ca-86f8-89d91e30de14"). InnerVolumeSpecName "kube-api-access-qs5cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:50.947116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:50.947086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf" (OuterVolumeSpecName: "kube-api-access-9g4pf") pod "bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" (UID: "bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f"). InnerVolumeSpecName "kube-api-access-9g4pf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:22:51.046278 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.046243    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs5cp\" (UniqueName: \"kubernetes.io/projected/f28024cc-5cf7-42ca-86f8-89d91e30de14-kube-api-access-qs5cp\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:51.046278 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.046270    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g4pf\" (UniqueName: \"kubernetes.io/projected/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f-kube-api-access-9g4pf\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:22:51.233060 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.233029    2576 generic.go:356] "Generic (PLEG): container finished" podID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" containerID="bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832" exitCode=0
Sep 06 05:22:51.233277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.233074    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"
Sep 06 05:22:51.233277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.233113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" event={"ID":"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f","Type":"ContainerDied","Data":"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"}
Sep 06 05:22:51.233277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.233194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v" event={"ID":"bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f","Type":"ContainerDied","Data":"f95f93946ebeeeb64b9389f1dd2ea4cf259ad85bfc2d509bc10b273f1ba601ba"}
Sep 06 05:22:51.233277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.233227    2576 scope.go:117] "RemoveContainer" containerID="bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"
Sep 06 05:22:51.234620 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.234594    2576 generic.go:356] "Generic (PLEG): container finished" podID="f28024cc-5cf7-42ca-86f8-89d91e30de14" containerID="ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb" exitCode=0
Sep 06 05:22:51.234747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.234638    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"
Sep 06 05:22:51.234747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.234638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" event={"ID":"f28024cc-5cf7-42ca-86f8-89d91e30de14","Type":"ContainerDied","Data":"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"}
Sep 06 05:22:51.234864 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.234762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk" event={"ID":"f28024cc-5cf7-42ca-86f8-89d91e30de14","Type":"ContainerDied","Data":"bf85a4ec3a958fa70569417e05c5808f4eafdec70140e92b31e4266003e662bf"}
Sep 06 05:22:51.242475 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.242458    2576 scope.go:117] "RemoveContainer" containerID="bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"
Sep 06 05:22:51.242761 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:51.242742    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832\": container with ID starting with bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832 not found: ID does not exist" containerID="bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"
Sep 06 05:22:51.242825 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.242767    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832"} err="failed to get container status \"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832\": rpc error: code = NotFound desc = could not find container \"bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832\": container with ID starting with bc8a9807ecf797e4909961bd0f3d5196c6bc4180776d370af269088118d06832 not found: ID does not exist"
Sep 06 05:22:51.242825 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.242785    2576 scope.go:117] "RemoveContainer" containerID="ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"
Sep 06 05:22:51.250945 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.250930    2576 scope.go:117] "RemoveContainer" containerID="ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"
Sep 06 05:22:51.251170 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:22:51.251153    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb\": container with ID starting with ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb not found: ID does not exist" containerID="ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"
Sep 06 05:22:51.251222 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.251176    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb"} err="failed to get container status \"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb\": rpc error: code = NotFound desc = could not find container \"ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb\": container with ID starting with ff84396005b719959cb194fd4e6a91f9343177f57c9a50f20615901a07478beb not found: ID does not exist"
Sep 06 05:22:51.256407 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.256387    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:22:51.259949 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.259929    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e24961-576cc979d6-fkfgk"]
Sep 06 05:22:51.269520 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.269498    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:22:51.272995 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:51.272976    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e24961-76b6596647-vgs4v"]
Sep 06 05:22:52.386460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:52.386428    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" path="/var/lib/kubelet/pods/bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f/volumes"
Sep 06 05:22:52.386871 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:52.386749    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf62b49-f92e-44dd-bcb7-42f816aadcc7" path="/var/lib/kubelet/pods/ddf62b49-f92e-44dd-bcb7-42f816aadcc7/volumes"
Sep 06 05:22:52.387102 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:52.387085    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f28024cc-5cf7-42ca-86f8-89d91e30de14" path="/var/lib/kubelet/pods/f28024cc-5cf7-42ca-86f8-89d91e30de14/volumes"
Sep 06 05:22:57.397031 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397003    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"]
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397352    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397363    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397373    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" containerName="payload-processing"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397379    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" containerName="payload-processing"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397387    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28024cc-5cf7-42ca-86f8-89d91e30de14" containerName="payload-pre-processing"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397393    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28024cc-5cf7-42ca-86f8-89d91e30de14" containerName="payload-pre-processing"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397400    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:22:57.397404 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397405    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397415    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf62b49-f92e-44dd-bcb7-42f816aadcc7" containerName="revoke-keys"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397420    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf62b49-f92e-44dd-bcb7-42f816aadcc7" containerName="revoke-keys"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397478    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf62b49-f92e-44dd-bcb7-42f816aadcc7" containerName="revoke-keys"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397486    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397493    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f28024cc-5cf7-42ca-86f8-89d91e30de14" containerName="payload-pre-processing"
Sep 06 05:22:57.397654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.397500    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf364c76-e22e-43ba-a3d2-b2a15a5ddc6f" containerName="payload-processing"
Sep 06 05:22:57.400559 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.400541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:22:57.403108 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.403088    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:22:57.414977 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.411696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"]
Sep 06 05:22:57.504586 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.504559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l\" (UID: \"29fb745b-f8f4-44ef-a20a-381831ea4dab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:22:57.606077 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.606045    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l\" (UID: \"29fb745b-f8f4-44ef-a20a-381831ea4dab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:22:57.606595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.606577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l\" (UID: \"29fb745b-f8f4-44ef-a20a-381831ea4dab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:22:57.720367 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.720327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:22:57.988939 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:57.988915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"]
Sep 06 05:22:57.992018 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:22:57.991991    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fb745b_f8f4_44ef_a20a_381831ea4dab.slice/crio-1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc WatchSource:0}: Error finding container 1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc: Status 404 returned error can't find the container with id 1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc
Sep 06 05:22:58.274470 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:58.274396    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l" event={"ID":"29fb745b-f8f4-44ef-a20a-381831ea4dab","Type":"ContainerStarted","Data":"29eb8e665c8665143d8c640eb6f3ca07ee5927b687d3c6156f10f842f9b04fcd"}
Sep 06 05:22:58.274470 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:58.274457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l" event={"ID":"29fb745b-f8f4-44ef-a20a-381831ea4dab","Type":"ContainerStarted","Data":"1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc"}
Sep 06 05:22:58.288701 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:58.288659    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l" podStartSLOduration=1.2886451270000001 podStartE2EDuration="1.288645127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:22:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:22:58.287477042 +0000 UTC m=+1640.376554736" watchObservedRunningTime="2026-09-06 05:22:58.288645127 +0000 UTC m=+1640.377722817"
Sep 06 05:22:59.280911 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:59.280875    2576 generic.go:356] "Generic (PLEG): container finished" podID="29fb745b-f8f4-44ef-a20a-381831ea4dab" containerID="29eb8e665c8665143d8c640eb6f3ca07ee5927b687d3c6156f10f842f9b04fcd" exitCode=0
Sep 06 05:22:59.281287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:22:59.280966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l" event={"ID":"29fb745b-f8f4-44ef-a20a-381831ea4dab","Type":"ContainerDied","Data":"29eb8e665c8665143d8c640eb6f3ca07ee5927b687d3c6156f10f842f9b04fcd"}
Sep 06 05:23:00.416368 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:00.416340    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:23:00.532553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:00.532523    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca\") pod \"29fb745b-f8f4-44ef-a20a-381831ea4dab\" (UID: \"29fb745b-f8f4-44ef-a20a-381831ea4dab\") "
Sep 06 05:23:00.532878 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:00.532858    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29fb745b-f8f4-44ef-a20a-381831ea4dab" (UID: "29fb745b-f8f4-44ef-a20a-381831ea4dab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:00.633335 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:00.633266    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29fb745b-f8f4-44ef-a20a-381831ea4dab-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:01.292233 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:01.292199    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"
Sep 06 05:23:01.292411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:01.292239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l" event={"ID":"29fb745b-f8f4-44ef-a20a-381831ea4dab","Type":"ContainerDied","Data":"1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc"}
Sep 06 05:23:01.292411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:01.292287    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a28b949a012db7dcdd74466bfd85bf7f755599c669f7138ac97bda7b8da7dcc"
Sep 06 05:23:08.017597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.017549    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:23:08.018331 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.018297    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" podUID="00935404-27ab-4dc1-8bf5-b992465e25a8" containerName="payload-pre-processing" containerID="cri-o://26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3" gracePeriod=30
Sep 06 05:23:08.150630 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.150592    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"]
Sep 06 05:23:08.152406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.152379    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-401f17-f759l"]
Sep 06 05:23:08.265478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.265452    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:23:08.328952 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.328858    2576 generic.go:356] "Generic (PLEG): container finished" podID="00935404-27ab-4dc1-8bf5-b992465e25a8" containerID="26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3" exitCode=0
Sep 06 05:23:08.329086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.328953    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"
Sep 06 05:23:08.329086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.328951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" event={"ID":"00935404-27ab-4dc1-8bf5-b992465e25a8","Type":"ContainerDied","Data":"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"}
Sep 06 05:23:08.329086 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.329075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd" event={"ID":"00935404-27ab-4dc1-8bf5-b992465e25a8","Type":"ContainerDied","Data":"8217584477e79c54b819dcfd61976f37514d4bb0bf881f914e1bc151e8f7653d"}
Sep 06 05:23:08.329196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.329098    2576 scope.go:117] "RemoveContainer" containerID="26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"
Sep 06 05:23:08.339244 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.339217    2576 scope.go:117] "RemoveContainer" containerID="26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"
Sep 06 05:23:08.339560 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:23:08.339538    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3\": container with ID starting with 26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3 not found: ID does not exist" containerID="26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"
Sep 06 05:23:08.339660 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.339563    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3"} err="failed to get container status \"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3\": rpc error: code = NotFound desc = could not find container \"26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3\": container with ID starting with 26de27e549eb4a9523de6229bdd2f8d970c0bddb9c89ef9043158240ff4c2aa3 not found: ID does not exist"
Sep 06 05:23:08.386203 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.386178    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29fb745b-f8f4-44ef-a20a-381831ea4dab" path="/var/lib/kubelet/pods/29fb745b-f8f4-44ef-a20a-381831ea4dab/volumes"
Sep 06 05:23:08.406833 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.406807    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9v5k\" (UniqueName: \"kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k\") pod \"00935404-27ab-4dc1-8bf5-b992465e25a8\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") "
Sep 06 05:23:08.407005 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.406936    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume\") pod \"00935404-27ab-4dc1-8bf5-b992465e25a8\" (UID: \"00935404-27ab-4dc1-8bf5-b992465e25a8\") "
Sep 06 05:23:08.407313 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.407285    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00935404-27ab-4dc1-8bf5-b992465e25a8" (UID: "00935404-27ab-4dc1-8bf5-b992465e25a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:08.409395 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.409344    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k" (OuterVolumeSpecName: "kube-api-access-q9v5k") pod "00935404-27ab-4dc1-8bf5-b992465e25a8" (UID: "00935404-27ab-4dc1-8bf5-b992465e25a8"). InnerVolumeSpecName "kube-api-access-q9v5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:08.514415 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.509602    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9v5k\" (UniqueName: \"kubernetes.io/projected/00935404-27ab-4dc1-8bf5-b992465e25a8-kube-api-access-q9v5k\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.514415 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.509636    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00935404-27ab-4dc1-8bf5-b992465e25a8-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:08.649585 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.649547    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:23:08.655702 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:08.655680    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-401f17-7679bc5ddb-zqjdd"]
Sep 06 05:23:10.386083 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:10.386013    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00935404-27ab-4dc1-8bf5-b992465e25a8" path="/var/lib/kubelet/pods/00935404-27ab-4dc1-8bf5-b992465e25a8/volumes"
Sep 06 05:23:14.307412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:14.307373    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:23:14.308038 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:14.307589    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy" containerID="cri-o://516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca" gracePeriod=30
Sep 06 05:23:19.561382 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.561360    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:23:19.616805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.616805 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616807    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616831    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616866    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmhvn\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616960    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617002 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.616992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617025    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617046    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:19.617255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617056    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data" (OuterVolumeSpecName: "istio-data") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:19.617255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617057    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs\") pod \"969f1593-db41-4205-bc6f-fc6689d0f881\" (UID: \"969f1593-db41-4205-bc6f-fc6689d0f881\") "
Sep 06 05:23:19.617438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617243    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:19.617438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617289    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:19.617438 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617411    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:23:19.617705 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617548    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.617705 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617702    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.617810 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617719    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.617810 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617735    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/969f1593-db41-4205-bc6f-fc6689d0f881-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.617810 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.617750    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.619135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.619113    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token" (OuterVolumeSpecName: "istio-token") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:19.619238 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.619215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn" (OuterVolumeSpecName: "kube-api-access-cmhvn") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "kube-api-access-cmhvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:23:19.619663 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.619645    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:23:19.619715 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.619658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "969f1593-db41-4205-bc6f-fc6689d0f881" (UID: "969f1593-db41-4205-bc6f-fc6689d0f881"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:23:19.718888 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.718848    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.718888 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.718881    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/969f1593-db41-4205-bc6f-fc6689d0f881-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.718888 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.718894    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmhvn\" (UniqueName: \"kubernetes.io/projected/969f1593-db41-4205-bc6f-fc6689d0f881-kube-api-access-cmhvn\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:19.719125 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:19.718933    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/969f1593-db41-4205-bc6f-fc6689d0f881-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:23:20.392016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.391980    2576 generic.go:356] "Generic (PLEG): container finished" podID="969f1593-db41-4205-bc6f-fc6689d0f881" containerID="516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca" exitCode=0
Sep 06 05:23:20.392209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.392032    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"
Sep 06 05:23:20.392209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.392062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerDied","Data":"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"}
Sep 06 05:23:20.392209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.392124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj" event={"ID":"969f1593-db41-4205-bc6f-fc6689d0f881","Type":"ContainerDied","Data":"654fbcb17a034b25ab420f9567cddcd74534f2476443e3b0a0315f4fa33cff81"}
Sep 06 05:23:20.392209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.392147    2576 scope.go:117] "RemoveContainer" containerID="516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"
Sep 06 05:23:20.402287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.402270    2576 scope.go:117] "RemoveContainer" containerID="9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"
Sep 06 05:23:20.411629 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.411610    2576 scope.go:117] "RemoveContainer" containerID="516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"
Sep 06 05:23:20.411724 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.411678    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:23:20.411932 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:23:20.411914    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca\": container with ID starting with 516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca not found: ID does not exist" containerID="516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"
Sep 06 05:23:20.412008 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.411941    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca"} err="failed to get container status \"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca\": rpc error: code = NotFound desc = could not find container \"516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca\": container with ID starting with 516fc5be7a25b655c0beadfcdc47236343bc619a0eb56b6eb447c777edcc5cca not found: ID does not exist"
Sep 06 05:23:20.412008 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.411959    2576 scope.go:117] "RemoveContainer" containerID="9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"
Sep 06 05:23:20.412243 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:23:20.412218    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c\": container with ID starting with 9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c not found: ID does not exist" containerID="9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"
Sep 06 05:23:20.412292 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.412249    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c"} err="failed to get container status \"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c\": rpc error: code = NotFound desc = could not find container \"9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c\": container with ID starting with 9298569eed3bff72e4c60dd7466a7c602d281ec2e4e39f753c0a7be8b7a2ab5c not found: ID does not exist"
Sep 06 05:23:20.416896 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:20.416867    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-401f17-openshift-default-65fff9fbb-g68xj"]
Sep 06 05:23:22.386147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:22.386113    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" path="/var/lib/kubelet/pods/969f1593-db41-4205-bc6f-fc6689d0f881/volumes"
Sep 06 05:23:27.680575 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680546    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680884    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29fb745b-f8f4-44ef-a20a-381831ea4dab" containerName="revoke-keys"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680895    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29fb745b-f8f4-44ef-a20a-381831ea4dab" containerName="revoke-keys"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680945    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680951    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680967    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00935404-27ab-4dc1-8bf5-b992465e25a8" containerName="payload-pre-processing"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680973    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="00935404-27ab-4dc1-8bf5-b992465e25a8" containerName="payload-pre-processing"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680979    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681029 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.680984    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.681041    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="00935404-27ab-4dc1-8bf5-b992465e25a8" containerName="payload-pre-processing"
Sep 06 05:23:27.681318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.681050    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.681057    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bd0409-26bb-47dd-8648-914393a30b61" containerName="istio-proxy"
Sep 06 05:23:27.681318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.681064    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="969f1593-db41-4205-bc6f-fc6689d0f881" containerName="istio-proxy"
Sep 06 05:23:27.681318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.681070    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="29fb745b-f8f4-44ef-a20a-381831ea4dab" containerName="revoke-keys"
Sep 06 05:23:27.685524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.685506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.688326 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.688304    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-41fda0-dockercfg-m9wks\""
Sep 06 05:23:27.688424 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.688307    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-41fda0\""
Sep 06 05:23:27.695652 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.695629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:23:27.716635 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.716614    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:23:27.719895 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.719882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.728013 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.727993    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:23:27.794760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.794725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.794957 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.794777    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p6xm\" (UniqueName: \"kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.794957 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.794938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8rl5\" (UniqueName: \"kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.795072 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.795010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.895928 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.895869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.896107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.895936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p6xm\" (UniqueName: \"kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.896107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.895982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8rl5\" (UniqueName: \"kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.896205 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.896171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.896829 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.896806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.896873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.896860    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.904683 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.904660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8rl5\" (UniqueName: \"kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5\") pod \"payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:27.904790 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.904667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p6xm\" (UniqueName: \"kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm\") pod \"payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:27.997164 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:27.997074    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:28.030601 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.030568    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:28.274657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.274631    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:23:28.276651 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:23:28.276622    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod891075de_e12b_42cc_ab25_8cbb72c8aa50.slice/crio-4b8a8f1d4d75f245884049184a8bb0320666ee4e3edb8673d3510eadb5c0408b WatchSource:0}: Error finding container 4b8a8f1d4d75f245884049184a8bb0320666ee4e3edb8673d3510eadb5c0408b: Status 404 returned error can't find the container with id 4b8a8f1d4d75f245884049184a8bb0320666ee4e3edb8673d3510eadb5c0408b
Sep 06 05:23:28.316711 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.316682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:23:28.317794 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:23:28.317767    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7dd667_3468_47a9_96c1_e6b86b57b3c4.slice/crio-c29a2b5790e16ca86203dabeeef695555daa93c07b3159b5201132ae07964a03 WatchSource:0}: Error finding container c29a2b5790e16ca86203dabeeef695555daa93c07b3159b5201132ae07964a03: Status 404 returned error can't find the container with id c29a2b5790e16ca86203dabeeef695555daa93c07b3159b5201132ae07964a03
Sep 06 05:23:28.433621 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.433568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" event={"ID":"891075de-e12b-42cc-ab25-8cbb72c8aa50","Type":"ContainerStarted","Data":"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"}
Sep 06 05:23:28.433813 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.433643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" event={"ID":"891075de-e12b-42cc-ab25-8cbb72c8aa50","Type":"ContainerStarted","Data":"4b8a8f1d4d75f245884049184a8bb0320666ee4e3edb8673d3510eadb5c0408b"}
Sep 06 05:23:28.433813 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.433678    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:28.435778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.435742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" event={"ID":"1c7dd667-3468-47a9-96c1-e6b86b57b3c4","Type":"ContainerStarted","Data":"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"}
Sep 06 05:23:28.435937 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.435789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" event={"ID":"1c7dd667-3468-47a9-96c1-e6b86b57b3c4","Type":"ContainerStarted","Data":"c29a2b5790e16ca86203dabeeef695555daa93c07b3159b5201132ae07964a03"}
Sep 06 05:23:28.435937 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.435811    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:28.452735 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.452691    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" podStartSLOduration=1.452676236 podStartE2EDuration="1.452676236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:28.450813615 +0000 UTC m=+1670.539891310" watchObservedRunningTime="2026-09-06 05:23:28.452676236 +0000 UTC m=+1670.541753928"
Sep 06 05:23:28.465169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:28.465128    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" podStartSLOduration=1.465113386 podStartE2EDuration="1.465113386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:28.464689152 +0000 UTC m=+1670.553766857" watchObservedRunningTime="2026-09-06 05:23:28.465113386 +0000 UTC m=+1670.554191078"
Sep 06 05:23:39.443849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:39.443819    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:23:39.444281 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:39.444102    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:23:51.594458 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.594423    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:23:51.601803 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.601776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.605536 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.605493    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-22321a-openshift-default-dockercfg-dbkdx\""
Sep 06 05:23:51.607381 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.607358    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:23:51.722539 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722719 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2swk\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722719 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722719 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722832 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722923 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722923 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.722923 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.722880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.823925 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.823871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.823935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.823955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.823974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824114    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2swk\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824599 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.824881 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.824863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.826472 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.826450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.826572 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.826515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.832209 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.832189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.832429 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.832409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2swk\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk\") pod \"e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") " pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:51.915447 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:51.915421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:52.189138 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.189114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:23:52.191175 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:23:52.191143    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0caadf45_c668_45b0_91fb_e5a057229870.slice/crio-07c62c5d3c627120ad58ca6cc2e810f9909d1b4bb5bfd80f0e56b39a230a6f5d WatchSource:0}: Error finding container 07c62c5d3c627120ad58ca6cc2e810f9909d1b4bb5bfd80f0e56b39a230a6f5d: Status 404 returned error can't find the container with id 07c62c5d3c627120ad58ca6cc2e810f9909d1b4bb5bfd80f0e56b39a230a6f5d
Sep 06 05:23:52.193502 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.193470    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:23:52.193581 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.193544    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:23:52.193581 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.193573    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:23:52.565540 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.565441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerStarted","Data":"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"}
Sep 06 05:23:52.565540 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.565508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerStarted","Data":"07c62c5d3c627120ad58ca6cc2e810f9909d1b4bb5bfd80f0e56b39a230a6f5d"}
Sep 06 05:23:52.587583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.587535    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podStartSLOduration=1.587518512 podStartE2EDuration="1.587518512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:52.586162372 +0000 UTC m=+1694.675240081" watchObservedRunningTime="2026-09-06 05:23:52.587518512 +0000 UTC m=+1694.676596204"
Sep 06 05:23:52.916568 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:52.916481    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:53.921033 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:53.921004    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:54.575063 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:54.575029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:54.576121 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:54.576103    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:23:56.194097 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.194062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:23:56.197697 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.197681    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.200130 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.200114    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-22321a-dockercfg-mp5xx\""
Sep 06 05:23:56.200219 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.200158    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-22321a\""
Sep 06 05:23:56.207337 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.207307    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:23:56.364828 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.364786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.365022 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.364840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns27r\" (UniqueName: \"kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.465744 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.465653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.465744 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.465712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns27r\" (UniqueName: \"kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.466453 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.466433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.473432 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.473405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns27r\" (UniqueName: \"kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r\") pod \"payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.509327 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.509298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:56.781551 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:56.781524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:23:56.784183 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:23:56.784150    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacd076e_7989_4f99_8b64_d546945a56c3.slice/crio-734ae49ba6afef2f194fb1d51e352eab24a734c28c46b08eb6f5816b20e6c73d WatchSource:0}: Error finding container 734ae49ba6afef2f194fb1d51e352eab24a734c28c46b08eb6f5816b20e6c73d: Status 404 returned error can't find the container with id 734ae49ba6afef2f194fb1d51e352eab24a734c28c46b08eb6f5816b20e6c73d
Sep 06 05:23:57.594979 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:57.594941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" event={"ID":"facd076e-7989-4f99-8b64-d546945a56c3","Type":"ContainerStarted","Data":"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"}
Sep 06 05:23:57.595433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:57.595004    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" event={"ID":"facd076e-7989-4f99-8b64-d546945a56c3","Type":"ContainerStarted","Data":"734ae49ba6afef2f194fb1d51e352eab24a734c28c46b08eb6f5816b20e6c73d"}
Sep 06 05:23:57.595433 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:57.595130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:23:57.612152 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:23:57.612102    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" podStartSLOduration=1.6120828230000002 podStartE2EDuration="1.612082823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:23:57.610039409 +0000 UTC m=+1699.699117103" watchObservedRunningTime="2026-09-06 05:23:57.612082823 +0000 UTC m=+1699.701160520"
Sep 06 05:24:08.601773 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:08.601745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:24:13.687813 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.687781    2576 generic.go:356] "Generic (PLEG): container finished" podID="0caadf45-c668-45b0-91fb-e5a057229870" containerID="aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a" exitCode=0
Sep 06 05:24:13.688301 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.687862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerDied","Data":"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"}
Sep 06 05:24:13.688354 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.688316    2576 scope.go:117] "RemoveContainer" containerID="aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"
Sep 06 05:24:13.689374 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.689343    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:24:13.689477 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.689455    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:24:13.689540 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:13.689502    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:24:14.697714 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:14.697678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerStarted","Data":"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"}
Sep 06 05:24:14.916631 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:14.916590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:24:14.916856 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:14.916832    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 06 05:24:14.916956 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:14.916927    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 06 05:24:15.916739 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:15.916705    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 06 05:24:15.917189 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:15.916775    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 06 05:24:16.915882 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:16.915846    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 06 05:24:16.916087 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:16.915938    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 06 05:24:17.915865 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:17.915826    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 06 05:24:17.916316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:17.915932    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 06 05:24:18.916044 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:18.916014    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 06 05:24:18.916473 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:18.916068    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 06 05:24:19.920261 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:19.920226    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:24:19.920660 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:19.920611    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:24:19.921186 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:19.921172    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:24:38.737115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:24:38.737076    2576 scope.go:117] "RemoveContainer" containerID="b0904e2220ee73717c479a3771907daff57a612032ea2d0d5595c9c89dfa14a3"
Sep 06 05:25:08.081287 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:08.081256    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v_1c7dd667-3468-47a9-96c1-e6b86b57b3c4/payload-processing/0.log"
Sep 06 05:25:09.004172 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.004138    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:09.006753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.006734    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.009153 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.009134    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-c0aaed-openshift-default-dockercfg-4m9kn\""
Sep 06 05:25:09.019521 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.019498    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:09.074345 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8n6t\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074490 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074670 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074670 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074670 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074670 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.074670 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.074662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8n6t\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175574 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.175861 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175804    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.176119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.175879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.176119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.176004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.176311 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.176295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.177912 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.177872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.178073 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.178058    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.182363 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.182341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.182628 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.182609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8n6t\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t\") pod \"e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") " pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.318318 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.318240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:09.588992 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.588969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:09.590582 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:25:09.590555    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55815332_8dc8_481e_a64d_ca4d99bf286c.slice/crio-35422ae341ef77030c001ab4f211780c68942df4b50f8f8859821308f61abcb3 WatchSource:0}: Error finding container 35422ae341ef77030c001ab4f211780c68942df4b50f8f8859821308f61abcb3: Status 404 returned error can't find the container with id 35422ae341ef77030c001ab4f211780c68942df4b50f8f8859821308f61abcb3
Sep 06 05:25:09.592789 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.592756    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:09.592888 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.592837    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:09.593016 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.592884    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:09.973217 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.973175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerStarted","Data":"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"}
Sep 06 05:25:09.973419 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.973228    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerStarted","Data":"35422ae341ef77030c001ab4f211780c68942df4b50f8f8859821308f61abcb3"}
Sep 06 05:25:09.991931 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:09.991855    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podStartSLOduration=1.9918410560000002 podStartE2EDuration="1.991841056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:25:09.990670768 +0000 UTC m=+1772.079748492" watchObservedRunningTime="2026-09-06 05:25:09.991841056 +0000 UTC m=+1772.080918750"
Sep 06 05:25:10.319194 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:10.319100    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:10.324340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:10.324316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:10.978581 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:10.978553    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:10.980092 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:10.980068    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:13.739768 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.739731    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:13.742484 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.742467    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.744926 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.744888    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-c0aaed\""
Sep 06 05:25:13.745041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.744962    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-c0aaed-dockercfg-c87vn\""
Sep 06 05:25:13.754074 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.754047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:13.811274 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.811246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.811434 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.811322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czxrz\" (UniqueName: \"kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.912140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.912106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.912319 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.912186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czxrz\" (UniqueName: \"kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.912807 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.912787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:13.919764 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:13.919748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czxrz\" (UniqueName: \"kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz\") pod \"payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:14.054610 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:14.054531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:14.323224 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:14.323193    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:14.325569 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:25:14.325541    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52b712e_9ca3_4528_8cb3_1b1613fd57d0.slice/crio-25ba2036b5d88ba9fac5e27757f678dff2b9c0c2586f295ad71445205b504017 WatchSource:0}: Error finding container 25ba2036b5d88ba9fac5e27757f678dff2b9c0c2586f295ad71445205b504017: Status 404 returned error can't find the container with id 25ba2036b5d88ba9fac5e27757f678dff2b9c0c2586f295ad71445205b504017
Sep 06 05:25:15.008849 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:15.008815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" event={"ID":"c52b712e-9ca3-4528-8cb3-1b1613fd57d0","Type":"ContainerStarted","Data":"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"}
Sep 06 05:25:15.009294 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:15.008870    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" event={"ID":"c52b712e-9ca3-4528-8cb3-1b1613fd57d0","Type":"ContainerStarted","Data":"25ba2036b5d88ba9fac5e27757f678dff2b9c0c2586f295ad71445205b504017"}
Sep 06 05:25:15.025982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:15.025934    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" podStartSLOduration=2.025919861 podStartE2EDuration="2.025919861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-06 05:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:25:15.023359992 +0000 UTC m=+1777.112437687" watchObservedRunningTime="2026-09-06 05:25:15.025919861 +0000 UTC m=+1777.114997575"
Sep 06 05:25:16.013498 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:16.013472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:27.020798 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:27.020768    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:31.098977 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.098946    2576 generic.go:356] "Generic (PLEG): container finished" podID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerID="a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195" exitCode=0
Sep 06 05:25:31.099412 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.099019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerDied","Data":"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"}
Sep 06 05:25:31.099470 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.099443    2576 scope.go:117] "RemoveContainer" containerID="a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"
Sep 06 05:25:31.100817 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.100726    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:31.100986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.100928    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:31.101116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:31.101052    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 06 05:25:32.105652 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:32.105612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerStarted","Data":"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"}
Sep 06 05:25:32.319140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:32.319098    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:32.319406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:32.319377    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:32.319480 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:32.319429    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:33.319688 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:33.319650    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:33.320212 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:33.319731    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:34.318831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:34.318785    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:34.319065 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:34.318863    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:35.319230 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:35.319197    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:35.319627 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:35.319258    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:36.318738 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:36.318708    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:36.318990 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:36.318966    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:37.319411 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.319375    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Sep 06 05:25:37.319887 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.319858    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Sep 06 05:25:37.561107 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.561072    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"]
Sep 06 05:25:37.563521 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.563505    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:37.565799 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.565778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:25:37.571158 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.571096    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"]
Sep 06 05:25:37.656930 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.656864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r\" (UID: \"7cf111dd-f2b5-4e15-80d0-792f0a484ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:37.758001 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.757960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r\" (UID: \"7cf111dd-f2b5-4e15-80d0-792f0a484ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:37.758631 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.758606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r\" (UID: \"7cf111dd-f2b5-4e15-80d0-792f0a484ada\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:37.878006 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:37.877877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:38.146140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.146034    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"]
Sep 06 05:25:38.149353 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:25:38.149323    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf111dd_f2b5_4e15_80d0_792f0a484ada.slice/crio-e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd WatchSource:0}: Error finding container e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd: Status 404 returned error can't find the container with id e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd
Sep 06 05:25:38.323293 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.323259    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:38.323731 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.323682    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:38.324513 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.324490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:38.476976 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.476949    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:25:38.481147 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.481128    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:25:38.484528 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.484510    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:25:38.488077 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:38.488061    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:25:39.145859 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:39.145825    2576 generic.go:356] "Generic (PLEG): container finished" podID="7cf111dd-f2b5-4e15-80d0-792f0a484ada" containerID="8ebd926eab5f292789ca9f614a59401f3c0fe80238e5fad535c38af3d8576ea1" exitCode=0
Sep 06 05:25:39.146115 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:39.146056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r" event={"ID":"7cf111dd-f2b5-4e15-80d0-792f0a484ada","Type":"ContainerDied","Data":"8ebd926eab5f292789ca9f614a59401f3c0fe80238e5fad535c38af3d8576ea1"}
Sep 06 05:25:39.146191 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:39.146128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r" event={"ID":"7cf111dd-f2b5-4e15-80d0-792f0a484ada","Type":"ContainerStarted","Data":"e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd"}
Sep 06 05:25:40.286330 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:40.286310    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:40.386077 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:40.386051    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca\") pod \"7cf111dd-f2b5-4e15-80d0-792f0a484ada\" (UID: \"7cf111dd-f2b5-4e15-80d0-792f0a484ada\") "
Sep 06 05:25:40.386389 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:40.386364    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7cf111dd-f2b5-4e15-80d0-792f0a484ada" (UID: "7cf111dd-f2b5-4e15-80d0-792f0a484ada"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:25:40.487390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:40.487356    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf111dd-f2b5-4e15-80d0-792f0a484ada-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:41.158045 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:41.158009    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"
Sep 06 05:25:41.158208 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:41.158007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r" event={"ID":"7cf111dd-f2b5-4e15-80d0-792f0a484ada","Type":"ContainerDied","Data":"e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd"}
Sep 06 05:25:41.158208 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:41.158134    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e39dc556621e0de1f72fc65a64534f708a99ea63cd7c3fae43420ae54b78cedd"
Sep 06 05:25:43.432949 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.432891    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:43.433420 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.433151    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" podUID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" containerName="payload-pre-processing" containerID="cri-o://c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8" gracePeriod=30
Sep 06 05:25:43.466700 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:25:43.466661    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52b712e_9ca3_4528_8cb3_1b1613fd57d0.slice/crio-conmon-c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8.scope\": RecentStats: unable to find data in memory cache]"
Sep 06 05:25:43.466939 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:25:43.466824    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52b712e_9ca3_4528_8cb3_1b1613fd57d0.slice/crio-conmon-c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52b712e_9ca3_4528_8cb3_1b1613fd57d0.slice/crio-c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8.scope\": RecentStats: unable to find data in memory cache]"
Sep 06 05:25:43.573361 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.573320    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"]
Sep 06 05:25:43.575289 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.575263    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c0aaed-fk75r"]
Sep 06 05:25:43.708856 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.708827    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:43.819789 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.819744    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume\") pod \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") "
Sep 06 05:25:43.820011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.819799    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czxrz\" (UniqueName: \"kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz\") pod \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\" (UID: \"c52b712e-9ca3-4528-8cb3-1b1613fd57d0\") "
Sep 06 05:25:43.820177 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.820153    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c52b712e-9ca3-4528-8cb3-1b1613fd57d0" (UID: "c52b712e-9ca3-4528-8cb3-1b1613fd57d0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:25:43.820300 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.820284    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:43.822119 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.822098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz" (OuterVolumeSpecName: "kube-api-access-czxrz") pod "c52b712e-9ca3-4528-8cb3-1b1613fd57d0" (UID: "c52b712e-9ca3-4528-8cb3-1b1613fd57d0"). InnerVolumeSpecName "kube-api-access-czxrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:25:43.920885 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:43.920852    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czxrz\" (UniqueName: \"kubernetes.io/projected/c52b712e-9ca3-4528-8cb3-1b1613fd57d0-kube-api-access-czxrz\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:44.175102 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.175070    2576 generic.go:356] "Generic (PLEG): container finished" podID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" containerID="c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8" exitCode=0
Sep 06 05:25:44.175277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.175123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"
Sep 06 05:25:44.175277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.175159    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" event={"ID":"c52b712e-9ca3-4528-8cb3-1b1613fd57d0","Type":"ContainerDied","Data":"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"}
Sep 06 05:25:44.175277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.175216    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn" event={"ID":"c52b712e-9ca3-4528-8cb3-1b1613fd57d0","Type":"ContainerDied","Data":"25ba2036b5d88ba9fac5e27757f678dff2b9c0c2586f295ad71445205b504017"}
Sep 06 05:25:44.175277 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.175240    2576 scope.go:117] "RemoveContainer" containerID="c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"
Sep 06 05:25:44.184406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.184389    2576 scope.go:117] "RemoveContainer" containerID="c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"
Sep 06 05:25:44.184670 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:25:44.184653    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8\": container with ID starting with c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8 not found: ID does not exist" containerID="c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"
Sep 06 05:25:44.184731 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.184678    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8"} err="failed to get container status \"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8\": rpc error: code = NotFound desc = could not find container \"c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8\": container with ID starting with c99b0a83b66dfc24d6d34affd635fb0ed93b5a3b98af225ff78bd1dff35ff0b8 not found: ID does not exist"
Sep 06 05:25:44.195879 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.195856    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:44.202117 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.202095    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c0aaed-69d5c854c8-x92xn"]
Sep 06 05:25:44.385839 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.385809    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf111dd-f2b5-4e15-80d0-792f0a484ada" path="/var/lib/kubelet/pods/7cf111dd-f2b5-4e15-80d0-792f0a484ada/volumes"
Sep 06 05:25:44.386273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:44.386257    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" path="/var/lib/kubelet/pods/c52b712e-9ca3-4528-8cb3-1b1613fd57d0/volumes"
Sep 06 05:25:49.824846 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:49.824808    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:49.825426 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:49.825081    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" containerID="cri-o://ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8" gracePeriod=30
Sep 06 05:25:53.325791 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.325758    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 06 05:25:53.326182 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.325814    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 06 05:25:53.470406 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470369    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"]
Sep 06 05:25:53.470783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470761    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf111dd-f2b5-4e15-80d0-792f0a484ada" containerName="revoke-keys"
Sep 06 05:25:53.470783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470780    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf111dd-f2b5-4e15-80d0-792f0a484ada" containerName="revoke-keys"
Sep 06 05:25:53.470981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470792    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" containerName="payload-pre-processing"
Sep 06 05:25:53.470981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470798    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" containerName="payload-pre-processing"
Sep 06 05:25:53.470981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470870    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf111dd-f2b5-4e15-80d0-792f0a484ada" containerName="revoke-keys"
Sep 06 05:25:53.470981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.470883    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c52b712e-9ca3-4528-8cb3-1b1613fd57d0" containerName="payload-pre-processing"
Sep 06 05:25:53.473565 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.473548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:53.475842 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.475813    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:25:53.479977 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.479952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"]
Sep 06 05:25:53.512379 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.512339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6\" (UID: \"d25a2626-9c2b-468a-83bd-373af97b7cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:53.613307 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.613218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6\" (UID: \"d25a2626-9c2b-468a-83bd-373af97b7cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:53.613863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.613840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6\" (UID: \"d25a2626-9c2b-468a-83bd-373af97b7cd9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:53.786373 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:53.786340    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:54.056211 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:54.056171    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"]
Sep 06 05:25:54.057812 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:25:54.057784    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25a2626_9c2b_468a_83bd_373af97b7cd9.slice/crio-d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca WatchSource:0}: Error finding container d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca: Status 404 returned error can't find the container with id d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca
Sep 06 05:25:54.236664 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:54.236635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6" event={"ID":"d25a2626-9c2b-468a-83bd-373af97b7cd9","Type":"ContainerStarted","Data":"d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca"}
Sep 06 05:25:55.072981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.072956    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:55.127211 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127229    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127274    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127346 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127292    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127357    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127398    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127447    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8n6t\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:25:55.127512 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data" (OuterVolumeSpecName: "istio-data") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:25:55.127760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy\") pod \"55815332-8dc8-481e-a64d-ca4d99bf286c\" (UID: \"55815332-8dc8-481e-a64d-ca4d99bf286c\") "
Sep 06 05:25:55.127760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127742    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127794    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127828    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.127989    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.128007    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.128016    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55815332-8dc8-481e-a64d-ca4d99bf286c-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.128025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.128027    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.128530 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.128040    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.129612 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.129586    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:25:55.129837 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.129817    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:25:55.130105 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.130084    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t" (OuterVolumeSpecName: "kube-api-access-d8n6t") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "kube-api-access-d8n6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:25:55.130169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.130098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token" (OuterVolumeSpecName: "istio-token") pod "55815332-8dc8-481e-a64d-ca4d99bf286c" (UID: "55815332-8dc8-481e-a64d-ca4d99bf286c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:25:55.228647 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.228622    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.228647 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.228647    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.228809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.228655    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.228809 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.228664    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8n6t\" (UniqueName: \"kubernetes.io/projected/55815332-8dc8-481e-a64d-ca4d99bf286c-kube-api-access-d8n6t\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:55.244341 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.244311    2576 generic.go:356] "Generic (PLEG): container finished" podID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerID="ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8" exitCode=0
Sep 06 05:25:55.244445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.244355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"
Sep 06 05:25:55.244445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.244394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerDied","Data":"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"}
Sep 06 05:25:55.244445 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.244440    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22" event={"ID":"55815332-8dc8-481e-a64d-ca4d99bf286c","Type":"ContainerDied","Data":"35422ae341ef77030c001ab4f211780c68942df4b50f8f8859821308f61abcb3"}
Sep 06 05:25:55.244583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.244471    2576 scope.go:117] "RemoveContainer" containerID="ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"
Sep 06 05:25:55.246061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.246026    2576 generic.go:356] "Generic (PLEG): container finished" podID="d25a2626-9c2b-468a-83bd-373af97b7cd9" containerID="c7de8318c5ff427d38fa29d4bf375b0f9baac6a1afa1cc5cbf5736d4285790fd" exitCode=0
Sep 06 05:25:55.246174 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.246152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6" event={"ID":"d25a2626-9c2b-468a-83bd-373af97b7cd9","Type":"ContainerDied","Data":"c7de8318c5ff427d38fa29d4bf375b0f9baac6a1afa1cc5cbf5736d4285790fd"}
Sep 06 05:25:55.254302 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.254286    2576 scope.go:117] "RemoveContainer" containerID="a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"
Sep 06 05:25:55.262865 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.262849    2576 scope.go:117] "RemoveContainer" containerID="ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"
Sep 06 05:25:55.263125 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:25:55.263109    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8\": container with ID starting with ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8 not found: ID does not exist" containerID="ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"
Sep 06 05:25:55.263189 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.263133    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8"} err="failed to get container status \"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8\": rpc error: code = NotFound desc = could not find container \"ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8\": container with ID starting with ff0b060d9e43f5080f4ac8345cb065bc9aa3c96cffb215880552e2a6f407ceb8 not found: ID does not exist"
Sep 06 05:25:55.263189 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.263149    2576 scope.go:117] "RemoveContainer" containerID="a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"
Sep 06 05:25:55.263395 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:25:55.263381    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195\": container with ID starting with a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195 not found: ID does not exist" containerID="a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"
Sep 06 05:25:55.263447 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.263398    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195"} err="failed to get container status \"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195\": rpc error: code = NotFound desc = could not find container \"a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195\": container with ID starting with a8d26692d296a9359e29458861b1ae72edda1e44ceea537523dd62dc1d228195 not found: ID does not exist"
Sep 06 05:25:55.273786 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.273766    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:55.278725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:55.278706    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c0aaed-openshift-default-58f669db56-d9m22"]
Sep 06 05:25:56.385893 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:56.385869    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" path="/var/lib/kubelet/pods/55815332-8dc8-481e-a64d-ca4d99bf286c/volumes"
Sep 06 05:25:56.386262 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:56.386067    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:56.439460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:56.439426    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca\") pod \"d25a2626-9c2b-468a-83bd-373af97b7cd9\" (UID: \"d25a2626-9c2b-468a-83bd-373af97b7cd9\") "
Sep 06 05:25:56.439753 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:56.439729    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d25a2626-9c2b-468a-83bd-373af97b7cd9" (UID: "d25a2626-9c2b-468a-83bd-373af97b7cd9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:25:56.540982 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:56.540892    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d25a2626-9c2b-468a-83bd-373af97b7cd9-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:25:57.258158 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:57.258128    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"
Sep 06 05:25:57.258340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:57.258164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6" event={"ID":"d25a2626-9c2b-468a-83bd-373af97b7cd9","Type":"ContainerDied","Data":"d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca"}
Sep 06 05:25:57.258340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:25:57.258213    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d755fed238c676108a54b17accf4b56ac766b20185bf74284127e75684b1cdca"
Sep 06 05:26:08.973672 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:08.973635    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:26:08.974300 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:08.973895    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" podUID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" containerName="payload-processing" containerID="cri-o://a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a" gracePeriod=30
Sep 06 05:26:08.983455 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:08.983424    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:26:08.983836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:08.983799    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" podUID="891075de-e12b-42cc-ab25-8cbb72c8aa50" containerName="payload-pre-processing" containerID="cri-o://04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2" gracePeriod=30
Sep 06 05:26:09.119747 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.119708    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"]
Sep 06 05:26:09.123252 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.123187    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-41fda0-xw6v6"]
Sep 06 05:26:09.268873 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.268852    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:26:09.271913 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.271877    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:26:09.323169 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.323133    2576 generic.go:356] "Generic (PLEG): container finished" podID="891075de-e12b-42cc-ab25-8cbb72c8aa50" containerID="04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2" exitCode=0
Sep 06 05:26:09.323390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.323191    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"
Sep 06 05:26:09.323390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.323226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" event={"ID":"891075de-e12b-42cc-ab25-8cbb72c8aa50","Type":"ContainerDied","Data":"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"}
Sep 06 05:26:09.323390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.323281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6" event={"ID":"891075de-e12b-42cc-ab25-8cbb72c8aa50","Type":"ContainerDied","Data":"4b8a8f1d4d75f245884049184a8bb0320666ee4e3edb8673d3510eadb5c0408b"}
Sep 06 05:26:09.323390 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.323303    2576 scope.go:117] "RemoveContainer" containerID="04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"
Sep 06 05:26:09.324859 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.324837    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" containerID="a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a" exitCode=0
Sep 06 05:26:09.325059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.324878    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"
Sep 06 05:26:09.325059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.324919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" event={"ID":"1c7dd667-3468-47a9-96c1-e6b86b57b3c4","Type":"ContainerDied","Data":"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"}
Sep 06 05:26:09.325059 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.324953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v" event={"ID":"1c7dd667-3468-47a9-96c1-e6b86b57b3c4","Type":"ContainerDied","Data":"c29a2b5790e16ca86203dabeeef695555daa93c07b3159b5201132ae07964a03"}
Sep 06 05:26:09.334424 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.334402    2576 scope.go:117] "RemoveContainer" containerID="04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"
Sep 06 05:26:09.334742 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:26:09.334714    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2\": container with ID starting with 04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2 not found: ID does not exist" containerID="04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"
Sep 06 05:26:09.334831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.334755    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2"} err="failed to get container status \"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2\": rpc error: code = NotFound desc = could not find container \"04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2\": container with ID starting with 04d4997335f73b63a5b810abbe38b05e7f3b8b916b65a6dad54d9d2b2fe37cf2 not found: ID does not exist"
Sep 06 05:26:09.334831 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.334778    2576 scope.go:117] "RemoveContainer" containerID="a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"
Sep 06 05:26:09.342888 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.342864    2576 scope.go:117] "RemoveContainer" containerID="a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"
Sep 06 05:26:09.343231 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:26:09.343212    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a\": container with ID starting with a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a not found: ID does not exist" containerID="a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"
Sep 06 05:26:09.343288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.343239    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a"} err="failed to get container status \"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a\": rpc error: code = NotFound desc = could not find container \"a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a\": container with ID starting with a71cb71fd318aee921009dc0519e246b4e392668a6f21ffca8d1e7be450b863a not found: ID does not exist"
Sep 06 05:26:09.350553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.350534    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8rl5\" (UniqueName: \"kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5\") pod \"891075de-e12b-42cc-ab25-8cbb72c8aa50\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") "
Sep 06 05:26:09.350657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.350593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p6xm\" (UniqueName: \"kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm\") pod \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") "
Sep 06 05:26:09.350657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.350642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume\") pod \"891075de-e12b-42cc-ab25-8cbb72c8aa50\" (UID: \"891075de-e12b-42cc-ab25-8cbb72c8aa50\") "
Sep 06 05:26:09.350772 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.350698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume\") pod \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\" (UID: \"1c7dd667-3468-47a9-96c1-e6b86b57b3c4\") "
Sep 06 05:26:09.351068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.351046    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "891075de-e12b-42cc-ab25-8cbb72c8aa50" (UID: "891075de-e12b-42cc-ab25-8cbb72c8aa50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:26:09.351143 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.351086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c7dd667-3468-47a9-96c1-e6b86b57b3c4" (UID: "1c7dd667-3468-47a9-96c1-e6b86b57b3c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:26:09.352835 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.352813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5" (OuterVolumeSpecName: "kube-api-access-n8rl5") pod "891075de-e12b-42cc-ab25-8cbb72c8aa50" (UID: "891075de-e12b-42cc-ab25-8cbb72c8aa50"). InnerVolumeSpecName "kube-api-access-n8rl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:26:09.352939 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.352918    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm" (OuterVolumeSpecName: "kube-api-access-7p6xm") pod "1c7dd667-3468-47a9-96c1-e6b86b57b3c4" (UID: "1c7dd667-3468-47a9-96c1-e6b86b57b3c4"). InnerVolumeSpecName "kube-api-access-7p6xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:26:09.451962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.451925    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8rl5\" (UniqueName: \"kubernetes.io/projected/891075de-e12b-42cc-ab25-8cbb72c8aa50-kube-api-access-n8rl5\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.451962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.451955    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p6xm\" (UniqueName: \"kubernetes.io/projected/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-kube-api-access-7p6xm\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.451962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.451965    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/891075de-e12b-42cc-ab25-8cbb72c8aa50-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.451962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.451974    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c7dd667-3468-47a9-96c1-e6b86b57b3c4-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:26:09.648012 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.647972    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:26:09.658138 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.658102    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-41fda0-7ff684b778-7fqf6"]
Sep 06 05:26:09.668440 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.668409    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:26:09.672360 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:09.672324    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-41fda0-6895fb7d85-wn98v"]
Sep 06 05:26:10.385536 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:10.385502    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" path="/var/lib/kubelet/pods/1c7dd667-3468-47a9-96c1-e6b86b57b3c4/volumes"
Sep 06 05:26:10.386090 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:10.386025    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="891075de-e12b-42cc-ab25-8cbb72c8aa50" path="/var/lib/kubelet/pods/891075de-e12b-42cc-ab25-8cbb72c8aa50/volumes"
Sep 06 05:26:10.386383 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:10.386364    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d25a2626-9c2b-468a-83bd-373af97b7cd9" path="/var/lib/kubelet/pods/d25a2626-9c2b-468a-83bd-373af97b7cd9/volumes"
Sep 06 05:26:57.809639 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.809600    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"]
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810038    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="891075de-e12b-42cc-ab25-8cbb72c8aa50" containerName="payload-pre-processing"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810051    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="891075de-e12b-42cc-ab25-8cbb72c8aa50" containerName="payload-pre-processing"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810058    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d25a2626-9c2b-468a-83bd-373af97b7cd9" containerName="revoke-keys"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810063    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d25a2626-9c2b-468a-83bd-373af97b7cd9" containerName="revoke-keys"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810078    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810084    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810097    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" containerName="payload-processing"
Sep 06 05:26:57.810116 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810103    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" containerName="payload-processing"
Sep 06 05:26:57.810370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810161    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:26:57.810370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810169    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7dd667-3468-47a9-96c1-e6b86b57b3c4" containerName="payload-processing"
Sep 06 05:26:57.810370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810174    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d25a2626-9c2b-468a-83bd-373af97b7cd9" containerName="revoke-keys"
Sep 06 05:26:57.810370 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.810182    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="891075de-e12b-42cc-ab25-8cbb72c8aa50" containerName="payload-pre-processing"
Sep 06 05:26:57.813366 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.813349    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:26:57.815649 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.815633    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:26:57.819483 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.819462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"]
Sep 06 05:26:57.900025 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:57.899984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4\" (UID: \"764bc643-edc0-4db2-83cb-90cb4fb6b3a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:26:58.001610 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:58.001572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4\" (UID: \"764bc643-edc0-4db2-83cb-90cb4fb6b3a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:26:58.002255 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:58.002230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4\" (UID: \"764bc643-edc0-4db2-83cb-90cb4fb6b3a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:26:58.125924 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:58.125800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:26:58.397234 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:58.397207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"]
Sep 06 05:26:58.399188 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:26:58.399159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764bc643_edc0_4db2_83cb_90cb4fb6b3a2.slice/crio-85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad WatchSource:0}: Error finding container 85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad: Status 404 returned error can't find the container with id 85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad
Sep 06 05:26:58.575616 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:58.575587    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4" event={"ID":"764bc643-edc0-4db2-83cb-90cb4fb6b3a2","Type":"ContainerStarted","Data":"85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad"}
Sep 06 05:26:59.582605 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:59.582572    2576 generic.go:356] "Generic (PLEG): container finished" podID="764bc643-edc0-4db2-83cb-90cb4fb6b3a2" containerID="d158a3bf660ee2d5e8ea42b4cd9916d865051b6da6c98cbc374dff5b1c56347d" exitCode=0
Sep 06 05:26:59.583053 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:26:59.582612    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4" event={"ID":"764bc643-edc0-4db2-83cb-90cb4fb6b3a2","Type":"ContainerDied","Data":"d158a3bf660ee2d5e8ea42b4cd9916d865051b6da6c98cbc374dff5b1c56347d"}
Sep 06 05:27:00.722584 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:00.722560    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:27:00.829127 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:00.829088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca\") pod \"764bc643-edc0-4db2-83cb-90cb4fb6b3a2\" (UID: \"764bc643-edc0-4db2-83cb-90cb4fb6b3a2\") "
Sep 06 05:27:00.829482 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:00.829460    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "764bc643-edc0-4db2-83cb-90cb4fb6b3a2" (UID: "764bc643-edc0-4db2-83cb-90cb4fb6b3a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:00.930369 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:00.930336    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/764bc643-edc0-4db2-83cb-90cb4fb6b3a2-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:01.593857 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:01.593825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4" event={"ID":"764bc643-edc0-4db2-83cb-90cb4fb6b3a2","Type":"ContainerDied","Data":"85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad"}
Sep 06 05:27:01.594061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:01.593872    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85555d5d4682a9f5da06e78eeee6294df4b8fa7629dcf7748424e67b452a06ad"
Sep 06 05:27:01.594061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:01.593837    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"
Sep 06 05:27:08.257026 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.256991    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:27:08.257782 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.257741    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" podUID="facd076e-7989-4f99-8b64-d546945a56c3" containerName="payload-pre-processing" containerID="cri-o://794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7" gracePeriod=30
Sep 06 05:27:08.398468 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.398433    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"]
Sep 06 05:27:08.402070 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.402041    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-22321a-rbxd4"]
Sep 06 05:27:08.523565 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.523536    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:27:08.611651 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.611617    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns27r\" (UniqueName: \"kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r\") pod \"facd076e-7989-4f99-8b64-d546945a56c3\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") "
Sep 06 05:27:08.611822 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.611671    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume\") pod \"facd076e-7989-4f99-8b64-d546945a56c3\" (UID: \"facd076e-7989-4f99-8b64-d546945a56c3\") "
Sep 06 05:27:08.612104 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.612076    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "facd076e-7989-4f99-8b64-d546945a56c3" (UID: "facd076e-7989-4f99-8b64-d546945a56c3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:08.613929 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.613889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r" (OuterVolumeSpecName: "kube-api-access-ns27r") pod "facd076e-7989-4f99-8b64-d546945a56c3" (UID: "facd076e-7989-4f99-8b64-d546945a56c3"). InnerVolumeSpecName "kube-api-access-ns27r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:08.630698 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.630666    2576 generic.go:356] "Generic (PLEG): container finished" podID="facd076e-7989-4f99-8b64-d546945a56c3" containerID="794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7" exitCode=0
Sep 06 05:27:08.630801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.630715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" event={"ID":"facd076e-7989-4f99-8b64-d546945a56c3","Type":"ContainerDied","Data":"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"}
Sep 06 05:27:08.630801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.630723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"
Sep 06 05:27:08.630801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.630757    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh" event={"ID":"facd076e-7989-4f99-8b64-d546945a56c3","Type":"ContainerDied","Data":"734ae49ba6afef2f194fb1d51e352eab24a734c28c46b08eb6f5816b20e6c73d"}
Sep 06 05:27:08.630801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.630778    2576 scope.go:117] "RemoveContainer" containerID="794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"
Sep 06 05:27:08.640894 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.640866    2576 scope.go:117] "RemoveContainer" containerID="794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"
Sep 06 05:27:08.641563 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:08.641541    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7\": container with ID starting with 794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7 not found: ID does not exist" containerID="794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"
Sep 06 05:27:08.641654 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.641570    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7"} err="failed to get container status \"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7\": rpc error: code = NotFound desc = could not find container \"794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7\": container with ID starting with 794c04892a8728a71a98bdabf4aeb261e15775b0d1336a266179f82d523dcbe7 not found: ID does not exist"
Sep 06 05:27:08.653862 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.653835    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:27:08.660962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.658317    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-22321a-6bcc8796f8-5q8zh"]
Sep 06 05:27:08.712478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.712444    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/facd076e-7989-4f99-8b64-d546945a56c3-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:08.712478 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:08.712469    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns27r\" (UniqueName: \"kubernetes.io/projected/facd076e-7989-4f99-8b64-d546945a56c3-kube-api-access-ns27r\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:10.386053 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:10.386008    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="764bc643-edc0-4db2-83cb-90cb4fb6b3a2" path="/var/lib/kubelet/pods/764bc643-edc0-4db2-83cb-90cb4fb6b3a2/volumes"
Sep 06 05:27:10.386553 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:10.386539    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="facd076e-7989-4f99-8b64-d546945a56c3" path="/var/lib/kubelet/pods/facd076e-7989-4f99-8b64-d546945a56c3/volumes"
Sep 06 05:27:14.614836 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.614794    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:27:14.615329 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.615068    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy" containerID="cri-o://5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4" gracePeriod=30
Sep 06 05:27:14.967196 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967164    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"]
Sep 06 05:27:14.967538 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967526    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:27:14.967583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967539    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:27:14.967583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967555    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="facd076e-7989-4f99-8b64-d546945a56c3" containerName="payload-pre-processing"
Sep 06 05:27:14.967583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967561    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="facd076e-7989-4f99-8b64-d546945a56c3" containerName="payload-pre-processing"
Sep 06 05:27:14.967583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967578    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="764bc643-edc0-4db2-83cb-90cb4fb6b3a2" containerName="revoke-keys"
Sep 06 05:27:14.967583 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967584    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="764bc643-edc0-4db2-83cb-90cb4fb6b3a2" containerName="revoke-keys"
Sep 06 05:27:14.967740 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967636    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55815332-8dc8-481e-a64d-ca4d99bf286c" containerName="istio-proxy"
Sep 06 05:27:14.967740 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967644    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="764bc643-edc0-4db2-83cb-90cb4fb6b3a2" containerName="revoke-keys"
Sep 06 05:27:14.967740 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.967652    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="facd076e-7989-4f99-8b64-d546945a56c3" containerName="payload-pre-processing"
Sep 06 05:27:14.970720 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.970705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:14.973457 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.973436    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:27:14.976614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:14.976590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"]
Sep 06 05:27:15.070071 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.070040    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7\" (UID: \"1d649831-d38f-4238-ad0e-44188d1f6d06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:15.170986 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.170953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7\" (UID: \"1d649831-d38f-4238-ad0e-44188d1f6d06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:15.171565 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.171546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7\" (UID: \"1d649831-d38f-4238-ad0e-44188d1f6d06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:15.282555 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.282473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:15.550624 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.550595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"]
Sep 06 05:27:15.553252 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:27:15.553227    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d649831_d38f_4238_ad0e_44188d1f6d06.slice/crio-505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c WatchSource:0}: Error finding container 505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c: Status 404 returned error can't find the container with id 505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c
Sep 06 05:27:15.666825 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:15.666798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7" event={"ID":"1d649831-d38f-4238-ad0e-44188d1f6d06","Type":"ContainerStarted","Data":"505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c"}
Sep 06 05:27:16.673536 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:16.673499    2576 generic.go:356] "Generic (PLEG): container finished" podID="1d649831-d38f-4238-ad0e-44188d1f6d06" containerID="d6d0437e5d05b025d3191a5fe7c207ffe851e22bf776182326e1ebf10a00770a" exitCode=0
Sep 06 05:27:16.673943 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:16.673585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7" event={"ID":"1d649831-d38f-4238-ad0e-44188d1f6d06","Type":"ContainerDied","Data":"d6d0437e5d05b025d3191a5fe7c207ffe851e22bf776182326e1ebf10a00770a"}
Sep 06 05:27:17.813597 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:17.813574    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:17.993434 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:17.993350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca\") pod \"1d649831-d38f-4238-ad0e-44188d1f6d06\" (UID: \"1d649831-d38f-4238-ad0e-44188d1f6d06\") "
Sep 06 05:27:17.993712 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:17.993687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1d649831-d38f-4238-ad0e-44188d1f6d06" (UID: "1d649831-d38f-4238-ad0e-44188d1f6d06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:18.094515 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:18.094479    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d649831-d38f-4238-ad0e-44188d1f6d06-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:18.684819 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:18.684794    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"
Sep 06 05:27:18.684819 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:18.684809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7" event={"ID":"1d649831-d38f-4238-ad0e-44188d1f6d06","Type":"ContainerDied","Data":"505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c"}
Sep 06 05:27:18.685082 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:18.684861    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="505d877b819c12ea5f1bc0718f29452488add1f6e9ba0e22a33e1e05517a427c"
Sep 06 05:27:19.871959 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:19.871890    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:27:20.010863 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.010835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.010870    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.010925    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2swk\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.010974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011032 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011019    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011194 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011065    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011194 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011138    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011194 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:20.011194 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011178    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011394 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011259    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:20.011394 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert\") pod \"0caadf45-c668-45b0-91fb-e5a057229870\" (UID: \"0caadf45-c668-45b0-91fb-e5a057229870\") "
Sep 06 05:27:20.011494 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011478    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:20.011555 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data" (OuterVolumeSpecName: "istio-data") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:20.011748 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011682    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.011748 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011711    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.011748 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011726    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.011748 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011757    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.012175 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.011988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:20.013407 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.013384    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:20.013506 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.013445    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token" (OuterVolumeSpecName: "istio-token") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:20.013752 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.013733    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk" (OuterVolumeSpecName: "kube-api-access-m2swk") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "kube-api-access-m2swk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:20.013847 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.013748    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0caadf45-c668-45b0-91fb-e5a057229870" (UID: "0caadf45-c668-45b0-91fb-e5a057229870"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:27:20.113001 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.112980    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0caadf45-c668-45b0-91fb-e5a057229870-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.113001 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.113000    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2swk\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-kube-api-access-m2swk\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.113152 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.113010    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0caadf45-c668-45b0-91fb-e5a057229870-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.113152 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.113018    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0caadf45-c668-45b0-91fb-e5a057229870-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.113152 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.113025    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0caadf45-c668-45b0-91fb-e5a057229870-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:20.697078 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.697049    2576 generic.go:356] "Generic (PLEG): container finished" podID="0caadf45-c668-45b0-91fb-e5a057229870" containerID="5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4" exitCode=0
Sep 06 05:27:20.697261 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.697099    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"
Sep 06 05:27:20.697261 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.697100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerDied","Data":"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"}
Sep 06 05:27:20.697261 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.697223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2" event={"ID":"0caadf45-c668-45b0-91fb-e5a057229870","Type":"ContainerDied","Data":"07c62c5d3c627120ad58ca6cc2e810f9909d1b4bb5bfd80f0e56b39a230a6f5d"}
Sep 06 05:27:20.697261 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.697250    2576 scope.go:117] "RemoveContainer" containerID="5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"
Sep 06 05:27:20.705955 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.705938    2576 scope.go:117] "RemoveContainer" containerID="aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"
Sep 06 05:27:20.715013 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.714992    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:27:20.715336 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.715320    2576 scope.go:117] "RemoveContainer" containerID="5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"
Sep 06 05:27:20.715604 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:20.715583    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4\": container with ID starting with 5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4 not found: ID does not exist" containerID="5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"
Sep 06 05:27:20.715682 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.715611    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4"} err="failed to get container status \"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4\": rpc error: code = NotFound desc = could not find container \"5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4\": container with ID starting with 5d18170335bc63d6ddeb377ee123b9b765326ba3fd7157b209ad93967adaafd4 not found: ID does not exist"
Sep 06 05:27:20.715682 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.715628    2576 scope.go:117] "RemoveContainer" containerID="aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"
Sep 06 05:27:20.715869 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:20.715853    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a\": container with ID starting with aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a not found: ID does not exist" containerID="aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"
Sep 06 05:27:20.715927 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.715877    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a"} err="failed to get container status \"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a\": rpc error: code = NotFound desc = could not find container \"aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a\": container with ID starting with aa92698f12b2a6f9c1e9815b8ee037e2aa81887d95637ad7d62d7c0aef67063a not found: ID does not exist"
Sep 06 05:27:20.719998 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:20.719973    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-22321a-openshift-default-5446bd749d-xd4l2"]
Sep 06 05:27:22.385464 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:22.385431    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0caadf45-c668-45b0-91fb-e5a057229870" path="/var/lib/kubelet/pods/0caadf45-c668-45b0-91fb-e5a057229870/volumes"
Sep 06 05:27:25.115890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.115840    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:27:25.116854 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.116768    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" podUID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" containerName="payload-pre-processing" containerID="cri-o://62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611" gracePeriod=30
Sep 06 05:27:25.253630 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.253602    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"]
Sep 06 05:27:25.256366 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.256339    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-494740-dfdf7"]
Sep 06 05:27:25.360521 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.360496    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:27:25.462265 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.462237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5v8gv\" (UniqueName: \"kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv\") pod \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") "
Sep 06 05:27:25.462451 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.462296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume\") pod \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\" (UID: \"1c998e34-8012-48c2-b10e-f2e7ca5f693a\") "
Sep 06 05:27:25.462690 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.462668    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c998e34-8012-48c2-b10e-f2e7ca5f693a" (UID: "1c998e34-8012-48c2-b10e-f2e7ca5f693a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:25.464544 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.464519    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv" (OuterVolumeSpecName: "kube-api-access-5v8gv") pod "1c998e34-8012-48c2-b10e-f2e7ca5f693a" (UID: "1c998e34-8012-48c2-b10e-f2e7ca5f693a"). InnerVolumeSpecName "kube-api-access-5v8gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:25.563202 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.563173    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c998e34-8012-48c2-b10e-f2e7ca5f693a-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:25.563202 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.563197    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5v8gv\" (UniqueName: \"kubernetes.io/projected/1c998e34-8012-48c2-b10e-f2e7ca5f693a-kube-api-access-5v8gv\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:25.724737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.724649    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" containerID="62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611" exitCode=0
Sep 06 05:27:25.724737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.724697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" event={"ID":"1c998e34-8012-48c2-b10e-f2e7ca5f693a","Type":"ContainerDied","Data":"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"}
Sep 06 05:27:25.724737 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.724701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"
Sep 06 05:27:25.725017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.724741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t" event={"ID":"1c998e34-8012-48c2-b10e-f2e7ca5f693a","Type":"ContainerDied","Data":"c9fdf80ad84010c38235d1e39b55088e086ab6afc87b82e993dd5a1ef52f64c0"}
Sep 06 05:27:25.725017 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.724762    2576 scope.go:117] "RemoveContainer" containerID="62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"
Sep 06 05:27:25.733947 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.733928    2576 scope.go:117] "RemoveContainer" containerID="62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"
Sep 06 05:27:25.734232 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:25.734213    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611\": container with ID starting with 62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611 not found: ID does not exist" containerID="62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"
Sep 06 05:27:25.734298 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.734240    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611"} err="failed to get container status \"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611\": rpc error: code = NotFound desc = could not find container \"62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611\": container with ID starting with 62131687a7fd11c2dac8be2c028400c73cf0b3b467c2d59ada1e537031769611 not found: ID does not exist"
Sep 06 05:27:25.744573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.744546    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:27:25.747643 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:25.747621    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-494740-7f7c74-zxp9t"]
Sep 06 05:27:26.386410 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:26.386371    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" path="/var/lib/kubelet/pods/1c998e34-8012-48c2-b10e-f2e7ca5f693a/volumes"
Sep 06 05:27:26.386932 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:26.386829    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d649831-d38f-4238-ad0e-44188d1f6d06" path="/var/lib/kubelet/pods/1d649831-d38f-4238-ad0e-44188d1f6d06/volumes"
Sep 06 05:27:32.225695 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.225660    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"]
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226043    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226058    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226077    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d649831-d38f-4238-ad0e-44188d1f6d06" containerName="revoke-keys"
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226083    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d649831-d38f-4238-ad0e-44188d1f6d06" containerName="revoke-keys"
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226099    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" containerName="payload-pre-processing"
Sep 06 05:27:32.226140 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226106    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" containerName="payload-pre-processing"
Sep 06 05:27:32.226364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226156    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c998e34-8012-48c2-b10e-f2e7ca5f693a" containerName="payload-pre-processing"
Sep 06 05:27:32.226364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226164    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:27:32.226364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226170    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:27:32.226364 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.226175    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d649831-d38f-4238-ad0e-44188d1f6d06" containerName="revoke-keys"
Sep 06 05:27:32.229221 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.229206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:32.231531 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.231508    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:27:32.235208 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.235186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"]
Sep 06 05:27:32.327573 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.327542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58\" (UID: \"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:32.428052 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.428021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58\" (UID: \"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:32.428608 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.428588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58\" (UID: \"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:32.541711 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.541637    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:32.810743 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:32.810720    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"]
Sep 06 05:27:32.812823 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:27:32.812799    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c80a88_4abc_4b9f_a72f_d6d1e7c62ad0.slice/crio-1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d WatchSource:0}: Error finding container 1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d: Status 404 returned error can't find the container with id 1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d
Sep 06 05:27:33.770456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:33.770365    2576 generic.go:356] "Generic (PLEG): container finished" podID="23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" containerID="7ce8d91674bdb0a10daafefb8bd23b8f438a1c942c2980fb93c1eafc51e4a7a7" exitCode=0
Sep 06 05:27:33.770456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:33.770401    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58" event={"ID":"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0","Type":"ContainerDied","Data":"7ce8d91674bdb0a10daafefb8bd23b8f438a1c942c2980fb93c1eafc51e4a7a7"}
Sep 06 05:27:33.770456 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:33.770439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58" event={"ID":"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0","Type":"ContainerStarted","Data":"1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d"}
Sep 06 05:27:34.908658 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:34.908635    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:34.950471 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:34.950446    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca\") pod \"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0\" (UID: \"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0\") "
Sep 06 05:27:34.950760 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:34.950739    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" (UID: "23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:35.052168 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:35.052102    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0-maas-api-service-ca\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:35.781266 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:35.781235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58" event={"ID":"23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0","Type":"ContainerDied","Data":"1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d"}
Sep 06 05:27:35.781508 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:35.781270    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"
Sep 06 05:27:35.781508 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:35.781283    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cae0db51ac3a8deb620a54b84e2161d5576fc6c2107d34e9600e105e0fa236d"
Sep 06 05:27:42.636157 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.636120    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:27:42.636558 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.636419    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" podUID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" containerName="payload-pre-processing" containerID="cri-o://2d45d71dca63428fb2924924569593e8ebb5fecbf91f71d3094ae191384f35d6" gracePeriod=30
Sep 06 05:27:42.778316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.778279    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"]
Sep 06 05:27:42.780962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.780866    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-167b5b-5kx58"]
Sep 06 05:27:42.820801 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.820772    2576 generic.go:356] "Generic (PLEG): container finished" podID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" containerID="2d45d71dca63428fb2924924569593e8ebb5fecbf91f71d3094ae191384f35d6" exitCode=0
Sep 06 05:27:42.820981 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.820846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" event={"ID":"0cd5b011-21a7-43a7-98c6-0caccf573d7d","Type":"ContainerDied","Data":"2d45d71dca63428fb2924924569593e8ebb5fecbf91f71d3094ae191384f35d6"}
Sep 06 05:27:42.888615 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:42.888548    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:27:43.022807 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.022771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mss2b\" (UniqueName: \"kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b\") pod \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") "
Sep 06 05:27:43.022999 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.022853    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume\") pod \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\" (UID: \"0cd5b011-21a7-43a7-98c6-0caccf573d7d\") "
Sep 06 05:27:43.023198 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.023172    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cd5b011-21a7-43a7-98c6-0caccf573d7d" (UID: "0cd5b011-21a7-43a7-98c6-0caccf573d7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:43.024919 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.024879    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b" (OuterVolumeSpecName: "kube-api-access-mss2b") pod "0cd5b011-21a7-43a7-98c6-0caccf573d7d" (UID: "0cd5b011-21a7-43a7-98c6-0caccf573d7d"). InnerVolumeSpecName "kube-api-access-mss2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:43.124286 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.124256    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd5b011-21a7-43a7-98c6-0caccf573d7d-plugins-config-volume\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:43.124286 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.124282    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mss2b\" (UniqueName: \"kubernetes.io/projected/0cd5b011-21a7-43a7-98c6-0caccf573d7d-kube-api-access-mss2b\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:43.827680 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.827648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq" event={"ID":"0cd5b011-21a7-43a7-98c6-0caccf573d7d","Type":"ContainerDied","Data":"6aaa64032a1699ffb10537d5741aac749aec16b37a35f3e47deb654b94bc72c0"}
Sep 06 05:27:43.828181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.827684    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"
Sep 06 05:27:43.828181 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.827703    2576 scope.go:117] "RemoveContainer" containerID="2d45d71dca63428fb2924924569593e8ebb5fecbf91f71d3094ae191384f35d6"
Sep 06 05:27:43.850409 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.850382    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:27:43.855545 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:43.855523    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-167b5b-545b94784b-qtrlq"]
Sep 06 05:27:44.386303 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:44.386263    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" path="/var/lib/kubelet/pods/0cd5b011-21a7-43a7-98c6-0caccf573d7d/volumes"
Sep 06 05:27:44.386687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:44.386668    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" path="/var/lib/kubelet/pods/23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0/volumes"
Sep 06 05:27:49.128440 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:49.128404    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:27:49.128951 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:49.128648    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy" containerID="cri-o://5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277" gracePeriod=30
Sep 06 05:27:54.382644 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.382620    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:27:54.531996 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.531963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.531996 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532037    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532056    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532074    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532116    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532133    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd2vl\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532245 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532150    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert\") pod \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\" (UID: \"2bb1d907-f2f2-4613-8e8a-bfc05ed36455\") "
Sep 06 05:27:54.532631 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532392    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data" (OuterVolumeSpecName: "istio-data") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:54.532631 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532442    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:54.532631 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532421    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:54.532806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532655    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-certs\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.532806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532675    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-data\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.532806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532691    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-credential-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.533044 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.533003    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:54.533044 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.532937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 06 05:27:54.534357 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.534334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 06 05:27:54.534759 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.534727    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 06 05:27:54.535068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.535048    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token" (OuterVolumeSpecName: "istio-token") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:54.535135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.535055    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl" (OuterVolumeSpecName: "kube-api-access-fd2vl") pod "2bb1d907-f2f2-4613-8e8a-bfc05ed36455" (UID: "2bb1d907-f2f2-4613-8e8a-bfc05ed36455"). InnerVolumeSpecName "kube-api-access-fd2vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 06 05:27:54.633708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633671    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd2vl\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-kube-api-access-fd2vl\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.633708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633701    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istiod-ca-cert\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.633708 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633714    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-podinfo\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.633962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633725    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-envoy\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.633962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633737    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-workload-socket\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.633962 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.633747    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bb1d907-f2f2-4613-8e8a-bfc05ed36455-istio-token\") on node \"ip-10-0-130-133.ec2.internal\" DevicePath \"\""
Sep 06 05:27:54.887993 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.887883    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerID="5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277" exitCode=0
Sep 06 05:27:54.887993 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.887940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerDied","Data":"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"}
Sep 06 05:27:54.888240 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.888003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq" event={"ID":"2bb1d907-f2f2-4613-8e8a-bfc05ed36455","Type":"ContainerDied","Data":"58c8d2534039c93c952ff77b1ebaf4b84e93f3ddd49f2c35f52bbc6216e90408"}
Sep 06 05:27:54.888240 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.888030    2576 scope.go:117] "RemoveContainer" containerID="5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"
Sep 06 05:27:54.888240 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.887953    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"
Sep 06 05:27:54.898062 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.898047    2576 scope.go:117] "RemoveContainer" containerID="999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"
Sep 06 05:27:54.905865 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.905850    2576 scope.go:117] "RemoveContainer" containerID="5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"
Sep 06 05:27:54.906175 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:54.906148    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277\": container with ID starting with 5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277 not found: ID does not exist" containerID="5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"
Sep 06 05:27:54.906246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.906190    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277"} err="failed to get container status \"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277\": rpc error: code = NotFound desc = could not find container \"5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277\": container with ID starting with 5eef8b0fb7d3236349659b5f98947e25fb332fd9f3fd54f3b1b6ed1939f7f277 not found: ID does not exist"
Sep 06 05:27:54.906246 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.906216    2576 scope.go:117] "RemoveContainer" containerID="999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"
Sep 06 05:27:54.906538 ip-10-0-130-133 kubenswrapper[2576]: E0906 05:27:54.906516    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107\": container with ID starting with 999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107 not found: ID does not exist" containerID="999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"
Sep 06 05:27:54.906614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.906548    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107"} err="failed to get container status \"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107\": rpc error: code = NotFound desc = could not find container \"999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107\": container with ID starting with 999dc192269ce1286e045cb1384fba8381eb8640c7fb3184405b9da8844af107 not found: ID does not exist"
Sep 06 05:27:54.909011 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.908989    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:27:54.914338 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:54.914315    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-167b5b-openshift-default-54d855d7b8-vfwzq"]
Sep 06 05:27:56.386061 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:27:56.386019    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" path="/var/lib/kubelet/pods/2bb1d907-f2f2-4613-8e8a-bfc05ed36455/volumes"
Sep 06 05:28:38.816291 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:28:38.816212    2576 scope.go:117] "RemoveContainer" containerID="752361d92089a2f35e73af88d23123f30adc7999094b39e904115b9d1a9b16b4"
Sep 06 05:28:38.824819 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:28:38.824800    2576 scope.go:117] "RemoveContainer" containerID="9ffb2003b7d33455fd3e762b73a6bb68a038ffeb17a538b69fb5d3103877dc18"
Sep 06 05:29:38.853973 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:29:38.853941    2576 scope.go:117] "RemoveContainer" containerID="29eb8e665c8665143d8c640eb6f3ca07ee5927b687d3c6156f10f842f9b04fcd"
Sep 06 05:29:38.862510 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:29:38.862492    2576 scope.go:117] "RemoveContainer" containerID="4316a7112f2dbe4e24d11fc1d60d7e436dfdba17f7af4ec1eaa883060fd7b95d"
Sep 06 05:30:00.128273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128183    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"]
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128512    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128524    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0caadf45-c668-45b0-91fb-e5a057229870" containerName="istio-proxy"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128538    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128543    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128553    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" containerName="payload-pre-processing"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128559    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" containerName="payload-pre-processing"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128578    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" containerName="revoke-keys"
Sep 06 05:30:00.128626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128583    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" containerName="revoke-keys"
Sep 06 05:30:00.128887 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128632    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23c80a88-4abc-4b9f-a72f-d6d1e7c62ad0" containerName="revoke-keys"
Sep 06 05:30:00.128887 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128639    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:30:00.128887 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128646    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:30:00.128887 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.128654    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd5b011-21a7-43a7-98c6-0caccf573d7d" containerName="payload-pre-processing"
Sep 06 05:30:00.131657 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.131642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"
Sep 06 05:30:00.134120 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.134095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9lqz9\""
Sep 06 05:30:00.138703 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.138251    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"]
Sep 06 05:30:00.141922 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.141878    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"
Sep 06 05:30:00.410614 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.410592    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"]
Sep 06 05:30:00.412830 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:30:00.412799    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5817ca8_6751_4081_be78_c1ec18d03015.slice/crio-943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865 WatchSource:0}: Error finding container 943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865: Status 404 returned error can't find the container with id 943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865
Sep 06 05:30:00.503144 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:00.503117    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87" event={"ID":"b5817ca8-6751-4081-be78-c1ec18d03015","Type":"ContainerStarted","Data":"943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865"}
Sep 06 05:30:01.509524 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:01.509487    2576 generic.go:356] "Generic (PLEG): container finished" podID="b5817ca8-6751-4081-be78-c1ec18d03015" containerID="4896646abf5245fab1f080ecfd27391a4fcb5bdac34641904fdbbb98bc11ebbb" exitCode=0
Sep 06 05:30:01.509883 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:01.509580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87" event={"ID":"b5817ca8-6751-4081-be78-c1ec18d03015","Type":"ContainerDied","Data":"4896646abf5245fab1f080ecfd27391a4fcb5bdac34641904fdbbb98bc11ebbb"}
Sep 06 05:30:02.650149 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:02.650127    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"
Sep 06 05:30:03.520584 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:03.520561    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87"
Sep 06 05:30:03.520745 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:03.520603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29811210-ljf87" event={"ID":"b5817ca8-6751-4081-be78-c1ec18d03015","Type":"ContainerDied","Data":"943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865"}
Sep 06 05:30:03.520745 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:03.520645    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="943bb03f0884a4ae98adaedb27f0761fc994d08d52049120dc7a768443fa0865"
Sep 06 05:30:38.505266 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:38.505241    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:30:38.512701 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:38.512683    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:30:38.515313 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:38.515296    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:30:38.519783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:30:38.519767    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:31:38.891014 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:31:38.890936    2576 scope.go:117] "RemoveContainer" containerID="8ebd926eab5f292789ca9f614a59401f3c0fe80238e5fad535c38af3d8576ea1"
Sep 06 05:32:38.910187 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:32:38.910156    2576 scope.go:117] "RemoveContainer" containerID="c7de8318c5ff427d38fa29d4bf375b0f9baac6a1afa1cc5cbf5736d4285790fd"
Sep 06 05:33:38.935514 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:38.935477    2576 scope.go:117] "RemoveContainer" containerID="7ce8d91674bdb0a10daafefb8bd23b8f438a1c942c2980fb93c1eafc51e4a7a7"
Sep 06 05:33:38.943778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:38.943750    2576 scope.go:117] "RemoveContainer" containerID="d158a3bf660ee2d5e8ea42b4cd9916d865051b6da6c98cbc374dff5b1c56347d"
Sep 06 05:33:38.951193 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:38.951172    2576 scope.go:117] "RemoveContainer" containerID="d6d0437e5d05b025d3191a5fe7c207ffe851e22bf776182326e1ebf10a00770a"
Sep 06 05:33:39.166595 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:39.166563    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7vmbq_2153bc10-8b55-40ae-9c7e-154f1b917990/server/0.log"
Sep 06 05:33:39.278008 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:39.277923    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q6txt_5f0ddd00-140c-4fbe-bb49-9957077cbcce/manager/0.log"
Sep 06 05:33:39.385949 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:39.385892    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-2sjbp_73be6ee7-d026-426c-9054-8a94967d1e10/copy-manifests/0.log"
Sep 06 05:33:39.397780 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:39.397754    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-2sjbp_73be6ee7-d026-426c-9054-8a94967d1e10/manager/0.log"
Sep 06 05:33:40.215043 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:40.215014    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29811195-djjfc_e6532632-9fee-48f3-9fca-2f7af9526aa4/cleanup/0.log"
Sep 06 05:33:40.322621 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:40.322593    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29811210-ljf87_b5817ca8-6751-4081-be78-c1ec18d03015/cleanup/0.log"
Sep 06 05:33:40.435938 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:40.435893    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hz7r6_24318767-9f33-4bbd-8bb2-f5022018de8b/postgres/0.log"
Sep 06 05:33:42.331394 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:42.331363    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h74bm_96182306-f539-43a9-9a55-aca1aa258667/discovery/0.log"
Sep 06 05:33:42.789757 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:42.789723    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cfhdf_af58991f-5113-45a2-879e-2baf61e00a52/payload-pre-processing/0.log"
Sep 06 05:33:43.021312 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:43.021272    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b77cb778-p6mrm_7062649b-61ff-45c2-8c19-3277489659bc/router/0.log"
Sep 06 05:33:43.380844 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:43.380817    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lrfc5_519d5c9c-5052-44e9-8bc3-3a39af35ab73/main/0.log"
Sep 06 05:33:43.490272 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:43.490245    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7nk25_bdeb179a-b7aa-4d4d-985b-fb1616129323/main/0.log"
Sep 06 05:33:43.816775 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:43.816745    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f8lnxl_522248f3-a4fa-461d-8649-a2f07b110152/main/0.log"
Sep 06 05:33:51.841160 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:51.841128    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8vxsx_9e3acc0d-6ef3-4709-a29a-79e9de3b703e/global-pull-secret-syncer/0.log"
Sep 06 05:33:52.099980 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:52.099893    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rkzgd_91727639-16b7-49f3-9277-5a6806eb3237/konnectivity-agent/0.log"
Sep 06 05:33:52.158772 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:52.158737    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-133.ec2.internal_6f9cfc38a9d54d6881403da441b0a834/haproxy/0.log"
Sep 06 05:33:58.677782 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:58.677754    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-kpp95_109e5587-86c1-487f-8166-f8e29284cfdf/cluster-monitoring-operator/0.log"
Sep 06 05:33:58.893199 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:58.893174    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/node-exporter/0.log"
Sep 06 05:33:58.919531 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:58.919506    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/kube-rbac-proxy/0.log"
Sep 06 05:33:58.936667 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:58.936605    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q2zb8_5ebdbc2a-07e3-4051-8081-65588f2803d3/init-textfile/0.log"
Sep 06 05:33:59.134525 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.134499    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/prometheus/0.log"
Sep 06 05:33:59.191267 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.191190    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/config-reloader/0.log"
Sep 06 05:33:59.214362 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.214335    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/thanos-sidecar/0.log"
Sep 06 05:33:59.233720 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.233698    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/kube-rbac-proxy-web/0.log"
Sep 06 05:33:59.251359 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.251343    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/kube-rbac-proxy/0.log"
Sep 06 05:33:59.270565 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.270540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/kube-rbac-proxy-thanos/0.log"
Sep 06 05:33:59.288773 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:33:59.288752    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_68ec2c41-7ba2-43f9-8f9e-8b04288dec23/init-config-reloader/0.log"
Sep 06 05:34:01.544550 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:01.544520    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/2.log"
Sep 06 05:34:01.548481 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:01.548460    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-t55rq_45d3705d-364e-4846-b3d0-e10876db24d1/console-operator/3.log"
Sep 06 05:34:02.159730 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.159694    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"]
Sep 06 05:34:02.160068 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.160054    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5817ca8-6751-4081-be78-c1ec18d03015" containerName="cleanup"
Sep 06 05:34:02.160129 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.160073    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5817ca8-6751-4081-be78-c1ec18d03015" containerName="cleanup"
Sep 06 05:34:02.160129 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.160091    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:34:02.160129 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.160096    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb1d907-f2f2-4613-8e8a-bfc05ed36455" containerName="istio-proxy"
Sep 06 05:34:02.160239 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.160168    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5817ca8-6751-4081-be78-c1ec18d03015" containerName="cleanup"
Sep 06 05:34:02.163273 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.163257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.165806 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.165785    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ns8q\"/\"kube-root-ca.crt\""
Sep 06 05:34:02.165955 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.165827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5ns8q\"/\"openshift-service-ca.crt\""
Sep 06 05:34:02.166778 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.166759    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5ns8q\"/\"default-dockercfg-p7szt\""
Sep 06 05:34:02.171860 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.171838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"]
Sep 06 05:34:02.300041 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.300010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-sys\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.300308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.300051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-lib-modules\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.300308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.300127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlnqm\" (UniqueName: \"kubernetes.io/projected/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-kube-api-access-qlnqm\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.300308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.300157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-podres\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.300308 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.300214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-proc\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401156 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-sys\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401168    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-lib-modules\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlnqm\" (UniqueName: \"kubernetes.io/projected/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-kube-api-access-qlnqm\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-sys\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-podres\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401342 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-lib-modules\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401568 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-proc\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401568 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-podres\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.401568 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.401471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-proc\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.408678 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.408653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlnqm\" (UniqueName: \"kubernetes.io/projected/c5aaa8de-4891-43e2-8520-98f2ebd3e2df-kube-api-access-qlnqm\") pod \"perf-node-gather-daemonset-mjlgl\" (UID: \"c5aaa8de-4891-43e2-8520-98f2ebd3e2df\") " pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.473914 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.473867    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:02.740853 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.740761    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"]
Sep 06 05:34:02.743226 ip-10-0-130-133 kubenswrapper[2576]: W0906 05:34:02.743194    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc5aaa8de_4891_43e2_8520_98f2ebd3e2df.slice/crio-0dd57268fe479ae24ce26f45a636d1b3c388159fde8eb3da6b31efb9f8d5b716 WatchSource:0}: Error finding container 0dd57268fe479ae24ce26f45a636d1b3c388159fde8eb3da6b31efb9f8d5b716: Status 404 returned error can't find the container with id 0dd57268fe479ae24ce26f45a636d1b3c388159fde8eb3da6b31efb9f8d5b716
Sep 06 05:34:02.794636 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:02.794615    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-gntt5_c580bd12-8c7c-434a-8a45-a003ca171311/volume-data-source-validator/0.log"
Sep 06 05:34:03.638808 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:03.638776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl" event={"ID":"c5aaa8de-4891-43e2-8520-98f2ebd3e2df","Type":"ContainerStarted","Data":"b60a621fa95579174c39a7ed1b64a3825bdc8725c0074d093462ad0977562082"}
Sep 06 05:34:03.639035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:03.638836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl" event={"ID":"c5aaa8de-4891-43e2-8520-98f2ebd3e2df","Type":"ContainerStarted","Data":"0dd57268fe479ae24ce26f45a636d1b3c388159fde8eb3da6b31efb9f8d5b716"}
Sep 06 05:34:03.639035 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:03.638965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:03.653725 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:03.653665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl" podStartSLOduration=1.653643287 podStartE2EDuration="1.653643287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-06 05:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-06 05:34:03.653637893 +0000 UTC m=+2305.742715591" watchObservedRunningTime="2026-09-06 05:34:03.653643287 +0000 UTC m=+2305.742720983"
Sep 06 05:34:04.030191 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:04.030167    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v47hr_12b770a7-9c8d-4f77-8d3a-0b8f98ad91db/dns/0.log"
Sep 06 05:34:04.047540 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:04.047518    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v47hr_12b770a7-9c8d-4f77-8d3a-0b8f98ad91db/kube-rbac-proxy/0.log"
Sep 06 05:34:04.089811 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:04.089781    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jwsh9_544f66cd-a01a-480b-aea0-23c82fc314c0/dns-node-resolver/0.log"
Sep 06 05:34:04.600462 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:04.600433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7689454c6c-qvsv8_592d3736-2b3a-4ee9-86ca-6c100cf66da2/registry/0.log"
Sep 06 05:34:04.667398 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:04.667376    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rgc5h_a452e6b1-85c2-485d-8d50-3434e01caae4/node-ca/0.log"
Sep 06 05:34:05.797870 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:05.797837    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h74bm_96182306-f539-43a9-9a55-aca1aa258667/discovery/0.log"
Sep 06 05:34:05.914687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:05.914659    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cfhdf_af58991f-5113-45a2-879e-2baf61e00a52/payload-pre-processing/0.log"
Sep 06 05:34:05.986288 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:05.986259    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b77cb778-p6mrm_7062649b-61ff-45c2-8c19-3277489659bc/router/0.log"
Sep 06 05:34:06.804754 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:06.804728    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hzl5_80a2d8d2-7870-4e16-aaee-c242d7ea3c0c/serve-healthcheck-canary/0.log"
Sep 06 05:34:07.269316 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:07.269282    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-gjfnp_1c23ef83-d706-4e78-92c7-2da117488b31/insights-operator/0.log"
Sep 06 05:34:07.270567 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:07.270547    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-gjfnp_1c23ef83-d706-4e78-92c7-2da117488b31/insights-operator/1.log"
Sep 06 05:34:07.290111 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:07.290090    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzvzj_29b7b8e6-29c0-4b88-aea8-4934920a6305/kube-rbac-proxy/0.log"
Sep 06 05:34:07.308480 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:07.308459    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzvzj_29b7b8e6-29c0-4b88-aea8-4934920a6305/exporter/0.log"
Sep 06 05:34:07.372958 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:07.372934    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jzvzj_29b7b8e6-29c0-4b88-aea8-4934920a6305/extractor/0.log"
Sep 06 05:34:09.653518 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:09.653491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5ns8q/perf-node-gather-daemonset-mjlgl"
Sep 06 05:34:10.008890 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:10.008862    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7vmbq_2153bc10-8b55-40ae-9c7e-154f1b917990/server/0.log"
Sep 06 05:34:10.081825 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:10.081799    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q6txt_5f0ddd00-140c-4fbe-bb49-9957077cbcce/manager/0.log"
Sep 06 05:34:10.154623 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:10.154595    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-2sjbp_73be6ee7-d026-426c-9054-8a94967d1e10/manager/0.log"
Sep 06 05:34:10.171464 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:10.171440    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-657c48fd97-2sjbp_73be6ee7-d026-426c-9054-8a94967d1e10/copy-manifests/0.log"
Sep 06 05:34:11.613804 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:11.613780    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7d8d45985c-vrr6q_dea21342-0bed-44b8-b7d7-f1c1b70221f4/manager/0.log"
Sep 06 05:34:13.131896 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:13.131867    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h74bm_96182306-f539-43a9-9a55-aca1aa258667/discovery/0.log"
Sep 06 05:34:13.220117 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:13.220091    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cfhdf_af58991f-5113-45a2-879e-2baf61e00a52/payload-pre-processing/0.log"
Sep 06 05:34:13.274216 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:13.274195    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b77cb778-p6mrm_7062649b-61ff-45c2-8c19-3277489659bc/router/0.log"
Sep 06 05:34:15.149811 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:15.149777    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-h74bm_96182306-f539-43a9-9a55-aca1aa258667/discovery/0.log"
Sep 06 05:34:15.245590 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:15.245562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cfhdf_af58991f-5113-45a2-879e-2baf61e00a52/payload-pre-processing/0.log"
Sep 06 05:34:15.306218 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:15.306190    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57b77cb778-p6mrm_7062649b-61ff-45c2-8c19-3277489659bc/router/0.log"
Sep 06 05:34:18.088460 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.088385    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7k4dl_7909525c-4e27-41e3-881f-8f7127d0b124/kube-multus/0.log"
Sep 06 05:34:18.369783 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.369702    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/kube-multus-additional-cni-plugins/0.log"
Sep 06 05:34:18.387971 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.387941    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/egress-router-binary-copy/0.log"
Sep 06 05:34:18.405754 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.405731    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/cni-plugins/0.log"
Sep 06 05:34:18.423827 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.423805    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/bond-cni-plugin/0.log"
Sep 06 05:34:18.441340 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.441312    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/routeoverride-cni/0.log"
Sep 06 05:34:18.458913 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.458886    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/whereabouts-cni-bincopy/0.log"
Sep 06 05:34:18.475578 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.475552    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pjrd9_2757f210-9c9b-4762-8631-9348f13dc03e/whereabouts-cni/0.log"
Sep 06 05:34:18.679687 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.679660    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r7t92_4f3f6321-5aaf-4656-ad4c-04109aeba114/network-metrics-daemon/0.log"
Sep 06 05:34:18.696834 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:18.696808    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r7t92_4f3f6321-5aaf-4656-ad4c-04109aeba114/kube-rbac-proxy/0.log"
Sep 06 05:34:19.667093 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.667064    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-controller/0.log"
Sep 06 05:34:19.684073 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.684047    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/1.log"
Sep 06 05:34:19.684219 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.684078    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovn-acl-logging/0.log"
Sep 06 05:34:19.703896 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.703847    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/kube-rbac-proxy-node/0.log"
Sep 06 05:34:19.723626 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.723598    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/kube-rbac-proxy-ovn-metrics/0.log"
Sep 06 05:34:19.743465 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.743433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/northd/0.log"
Sep 06 05:34:19.764135 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.764107    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/nbdb/0.log"
Sep 06 05:34:19.782603 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.782562    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/sbdb/0.log"
Sep 06 05:34:19.840393 ip-10-0-130-133 kubenswrapper[2576]: I0906 05:34:19.840361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pmmpb_d3e510ba-cb0a-4e68-8ea6-39fb8ba3f87c/ovnkube-controller/0.log"