Sep 08 13:28:36.572633 ip-10-0-129-230 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 13:28:37.001958 ip-10-0-129-230 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:37.001958 ip-10-0-129-230 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 13:28:37.001958 ip-10-0-129-230 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:37.001958 ip-10-0-129-230 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 13:28:37.001958 ip-10-0-129-230 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:37.004780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.004696    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012361    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012381    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012385    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012391    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012395    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.012389 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012398    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012401    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012404    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012407    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012410    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012412    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012415    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012417    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012420    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012422    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012425    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012427    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012430    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012432    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012435    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012437    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012440    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012443    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012445    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.012622 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012447    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012450    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012452    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012455    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012457    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012459    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012462    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012468    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012470    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012472    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012475    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012478    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012481    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012483    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012486    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012489    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012492    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012495    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012497    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012499    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.013084 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012502    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012506    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012511    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012514    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012517    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012520    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012523    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012526    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012528    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012531    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012534    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012536    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012539    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012541    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012543    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012546    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012548    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012551    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012554    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.013590 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012556    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012559    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012561    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012564    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012567    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012569    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012572    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012574    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012577    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012580    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012582    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012585    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012588    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012591    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012594    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012596    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012599    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012601    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012604    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012606    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012609    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.014052 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012612    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012614    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012617    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012619    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012622    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012625    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012627    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012630    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012632    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012635    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012637    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012640    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012643    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012645    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.012647    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013305    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013312    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013314    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013317    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013320    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.014574 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013323    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013325    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013328    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013331    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013334    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013337    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013357    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013360    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013363    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013365    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013367    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013370    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013373    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013376    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013378    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013380    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013383    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013386    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013388    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013391    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.015073 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013393    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013396    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013398    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013403    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013406    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013409    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013412    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013416    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013419    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013422    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013425    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013429    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013432    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013435    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013437    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013440    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013443    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013445    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013448    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.015623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013450    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013453    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013455    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013458    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013460    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013463    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013466    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013469    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013471    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013474    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013476    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013479    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013483    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013486    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013488    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013491    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013493    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013496    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013499    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.016099 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013501    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013504    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013506    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013509    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013511    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013514    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013516    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013519    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013521    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013524    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013526    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013528    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013531    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013533    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013536    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013538    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013540    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013543    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013545    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013548    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.016569 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013550    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013553    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013556    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013558    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013560    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013563    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013565    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013568    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013570    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013572    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013575    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013578    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013580    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013583    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013585    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013588    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013986    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013992    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013995    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.013998    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.017075 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014001    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014004    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014006    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014009    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014011    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014014    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014016    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014019    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014021    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014024    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014026    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014028    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014031    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014034    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014037    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014039    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014044    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014047    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014050    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.017592 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014053    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014055    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014058    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014061    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014064    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014069    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014072    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014075    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014077    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014079    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014082    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014086    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014089    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014092    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014095    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014097    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014100    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014103    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014105    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014108    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.018053 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014110    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014113    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014115    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014118    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014120    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014123    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014126    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014129    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014131    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014135    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014137    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014140    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014143    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014145    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014149    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014151    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014153    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014156    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014158    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014161    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.018562 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014163    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014166    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014168    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014170    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014173    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014175    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014177    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014180    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014182    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014185    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014187    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014189    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014192    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014194    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014197    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014199    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014201    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014204    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014206    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014209    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.019062 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014212    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014214    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014217    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014219    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014222    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014225    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014227    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014229    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014232    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014234    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014237    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014239    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014252    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014256    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014259    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014262    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014325    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014372    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014378    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014383    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014387    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 13:28:37.019563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014390    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014394    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014398    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014401    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014407    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014411    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014414    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014417    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014419    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014422    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014425    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014428    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014431    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014435    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014439    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014442    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014445    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014448    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014451    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014455    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014457    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014461    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014463    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014466    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 13:28:37.020074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014469    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014472    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014475    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014479    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014482    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014484    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014487    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014490    2577 flags.go:64] FLAG: --enable-server="true"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014492    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014496    2577 flags.go:64] FLAG: --event-burst="100"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014499    2577 flags.go:64] FLAG: --event-qps="50"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014502    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014506    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014509    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014512    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014515    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014518    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014520    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014523    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014526    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014529    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014532    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014536    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014539    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014541    2577 flags.go:64] FLAG: --feature-gates=""
Sep 08 13:28:37.020700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014545    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014547    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014550    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014553    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014556    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014559    2577 flags.go:64] FLAG: --help="false"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014561    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014564    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014567    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014570    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014573    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014577    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014579    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014583    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014585    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014598    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014601    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014605    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014607    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014611    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014614    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014617    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014620    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014623    2577 flags.go:64] FLAG: --lock-file=""
Sep 08 13:28:37.021467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014625    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014628    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014631    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014636    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014639    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014643    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014646    2577 flags.go:64] FLAG: --logging-format="text"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014648    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014651    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014654    2577 flags.go:64] FLAG: --manifest-url=""
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014657    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014661    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014664    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014668    2577 flags.go:64] FLAG: --max-pods="110"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014671    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014673    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014676    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014679    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014682    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014684    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014687    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014694    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014697    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014700    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 13:28:37.022060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014702    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014706    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014711    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014713    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014717    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014720    2577 flags.go:64] FLAG: --port="10250"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014723    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014726    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe7da84283f903b7"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014729    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014732    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014734    2577 flags.go:64] FLAG: --register-node="true"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014737    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014741    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014745    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014748    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014751    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014754    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014757    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014760    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014763    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014765    2577 flags.go:64] FLAG: --runonce="false"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014768    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014771    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014774    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014776    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 13:28:37.022701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014779    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014782    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014785    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014788    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014790    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014793    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014796    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014798    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014801    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014804    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014806    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014813    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014815    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014818    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014822    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014825    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014827    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014830    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014833    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014835    2577 flags.go:64] FLAG: --v="2"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014841    2577 flags.go:64] FLAG: --version="false"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014845    2577 flags.go:64] FLAG: --vmodule=""
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014848    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.014851    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014964    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.023304 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014968    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014971    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014974    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014977    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014979    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014982    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014984    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014987    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014990    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014992    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.014996    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015000    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015003    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015006    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015009    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015012    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015015    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015017    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015021    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.023954 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015024    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015027    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015030    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015033    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015036    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015038    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015041    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015043    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015047    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015050    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015053    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015056    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015058    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015060    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015063    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015065    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015067    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015070    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015072    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015075    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.024449 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015077    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015080    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015082    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015085    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015088    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015090    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015092    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015095    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015097    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015099    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015102    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015106    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015108    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015111    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015113    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015116    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015118    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015120    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015123    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015125    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.024929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015129    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015131    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015134    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015137    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015139    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015141    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015144    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015146    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015149    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015151    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015154    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015156    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015158    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015161    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015163    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015165    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015168    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015171    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015175    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.025443 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015178    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015181    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015184    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015186    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015190    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015193    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015195    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015198    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015200    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015203    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015205    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015208    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015211    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015215    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015218    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015220    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015222    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015225    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015228    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.025915 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.015230    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.015236    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.022935    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023056    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023075    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023084    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023142    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023148    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023152    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023157    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023160    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023164    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023167    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023170    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.026406 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023172    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023175    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023178    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023180    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023184    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023188    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023191    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023193    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023196    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023199    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023202    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023204    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023207    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023209    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023212    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023214    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023217    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023219    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023222    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.026778 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023225    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023228    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023233    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023236    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023239    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023241    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023244    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023247    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023249    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023251    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023254    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023256    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023259    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023261    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023264    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023266    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023269    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023271    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023274    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023277    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.027241 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023279    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023281    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023284    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023286    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023289    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023291    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023294    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023296    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023299    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023301    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023303    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023306    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023309    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023311    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023314    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023317    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023320    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023323    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023325    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023327    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.027793 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023330    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023332    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023335    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023337    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023354    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023357    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023360    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023362    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023365    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023367    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023370    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023372    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023375    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023378    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023380    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023383    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023385    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023388    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023391    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023393    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.028281 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023396    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023398    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023401    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023403    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023406    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023408    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023412    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023415    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023418    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023421    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023424    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023426    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023431    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023541    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023545    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:37.028782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023548    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023551    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023554    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023557    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023560    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023563    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023565    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023567    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023570    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023573    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023576    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023578    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023581    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023583    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023586    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023590    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023593    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023596    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023598    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:37.029151 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023601    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023604    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023606    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023610    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023612    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023615    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023618    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023621    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023623    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023626    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023628    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023631    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023633    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023636    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023638    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023640    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023643    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023645    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023648    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023650    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:37.029688 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023653    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023655    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023657    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023660    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023662    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023665    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023667    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023670    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023672    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023675    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023677    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023679    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023682    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023685    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023687    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023690    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023692    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023695    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023697    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023700    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:37.030162 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023703    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023706    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023708    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023710    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023713    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023715    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023718    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023720    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023723    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023726    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023728    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023731    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023733    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023736    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023738    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023741    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023743    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023746    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023748    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023751    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:37.030696 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023753    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023755    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023759    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023762    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023766    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023769    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023771    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023775    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023777    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023780    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023783    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023785    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023789    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023792    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023795    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023798    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023801    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:37.031168 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:37.023804    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023809    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.023914    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.026270    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.027318    2577 server.go:1019] "Starting client certificate rotation"
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.027379    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.028223    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:28:37.031617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.030166    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:28:37.032145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.032121    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:28:37.044229 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.044211    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 08 13:28:37.047503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.047486    2577 log.go:25] "Validated CRI v1 image API"
Sep 08 13:28:37.048315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.048301    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 13:28:37.055284 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.055257    2577 fs.go:135] Filesystem UUIDs: map[2987aa68-9d16-4694-b3c9-91ab25495e2a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8df24116-4be8-47be-b279-a9ff18f26123:/dev/nvme0n1p3]
Sep 08 13:28:37.055394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.055283    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 13:28:37.056703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.056682    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:28:37.060446 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.060314    2577 manager.go:216] Machine: {Timestamp:2026-09-08 13:28:37.058934985 +0000 UTC m=+0.375338540 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100771 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2048ba19b9a6536dfb342c52de30bd SystemUUID:ec2048ba-19b9-a653-6dfb-342c52de30bd BootID:40844fb5-4d65-4493-a0fa-fa6c4235a7cc Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c5:2c:e4:05:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c5:2c:e4:05:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:97:c2:a8:3b:27 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 13:28:37.060446 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.060436    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 13:28:37.060597 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.060570    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 13:28:37.064147 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.064110    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 13:28:37.064310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.064149    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 13:28:37.065070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.065058    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 13:28:37.065121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.065076    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 13:28:37.065121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.065086    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:28:37.065769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.065757    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:28:37.067578 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.067566    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 13:28:37.070963 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.070949    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:28:37.071228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071216    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 13:28:37.071285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071254    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 13:28:37.071285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071269    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 13:28:37.071408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071291    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 13:28:37.071408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071305    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 08 13:28:37.071408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071319    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 13:28:37.071716 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071703    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:28:37.071775 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.071726    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:28:37.075721 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.075705    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 13:28:37.078618 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078603    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 13:28:37.078699 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078640    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 13:28:37.078834 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078822    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 13:28:37.078892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078843    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 13:28:37.078892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078852    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 13:28:37.078892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078860    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 13:28:37.078892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078872    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 13:28:37.078892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.078880    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 13:28:37.079545 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079534    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 13:28:37.079603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079547    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 13:28:37.079603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079557    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 13:28:37.079603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079566    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 13:28:37.079603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079578    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 13:28:37.079603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.079590    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 13:28:37.080525 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.080513    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 13:28:37.080574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.080528    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 13:28:37.081151 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.081113    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 13:28:37.081184 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.081104    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 13:28:37.084204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.084189    2577 server.go:1268] "Started kubelet"
Sep 08 13:28:37.084275 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.084224    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 13:28:37.084816 ip-10-0-129-230 systemd[1]: Started Kubernetes Kubelet.
Sep 08 13:28:37.085020 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.084940    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 13:28:37.085079 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.085045    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 13:28:37.086936 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.086921    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 13:28:37.089111 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.089049    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 13:28:37.091190 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.091164    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 13:28:37.092604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.092588    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 13:28:37.094000 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.092697    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-230.ec2.internal.18d35b8818b12c98  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-230.ec2.internal,UID:ip-10-0-129-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-230.ec2.internal,},FirstTimestamp:2026-09-08 13:28:37.084163224 +0000 UTC m=+0.400566769,LastTimestamp:2026-09-08 13:28:37.084163224 +0000 UTC m=+0.400566769,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-230.ec2.internal,}"
Sep 08 13:28:37.095660 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.095643    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 13:28:37.096377 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.096362    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 13:28:37.096494 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.096470    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.097122 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097108    2577 factory.go:55] Registering systemd factory
Sep 08 13:28:37.097204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097141    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 08 13:28:37.097204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097168    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 13:28:37.097204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097171    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 13:28:37.097204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097192    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 13:28:37.097377 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097265    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 13:28:37.097377 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097275    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 13:28:37.097444 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097394    2577 factory.go:153] Registering CRI-O factory
Sep 08 13:28:37.097444 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097404    2577 factory.go:223] Registration of the crio container factory successfully
Sep 08 13:28:37.097500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097455    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 13:28:37.097500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097474    2577 factory.go:103] Registering Raw factory
Sep 08 13:28:37.097500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097489    2577 manager.go:1196] Started watching for new ooms in manager
Sep 08 13:28:37.097904 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.097893    2577 manager.go:349] Starting recovery of all containers
Sep 08 13:28:37.098291 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.098272    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 13:28:37.098400 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.098382    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 13:28:37.105869 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.105825    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 13:28:37.109158 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.109138    2577 manager.go:354] Recovery completed
Sep 08 13:28:37.114142 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.114129    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.115884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.115855    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.115954 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.115899    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.115954 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.115913    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.116526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.116508    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 13:28:37.116526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.116520    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.116638 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.116522    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 13:28:37.116638 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.116583    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:28:37.117219 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117205    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.117273 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117229    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.117273 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117239    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.117582 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117566    2577 policy_none.go:49] "None policy: Start"
Sep 08 13:28:37.117644 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117588    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 13:28:37.117644 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.117602    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 13:28:37.118534 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.118523    2577 policy_none.go:47] "Start"
Sep 08 13:28:37.125668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.125646    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt2n9"
Sep 08 13:28:37.125963 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.125899    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-230.ec2.internal.18d35b881a952cd5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-230.ec2.internal,UID:ip-10-0-129-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-230.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-230.ec2.internal,},FirstTimestamp:2026-09-08 13:28:37.115882709 +0000 UTC m=+0.432286253,LastTimestamp:2026-09-08 13:28:37.115882709 +0000 UTC m=+0.432286253,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-230.ec2.internal,}"
Sep 08 13:28:37.133030 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.133015    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pt2n9"
Sep 08 13:28:37.133932 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.133870    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-230.ec2.internal.18d35b881a958aef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-230.ec2.internal,UID:ip-10-0-129-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-230.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-230.ec2.internal,},FirstTimestamp:2026-09-08 13:28:37.115906799 +0000 UTC m=+0.432310343,LastTimestamp:2026-09-08 13:28:37.115906799 +0000 UTC m=+0.432310343,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-230.ec2.internal,}"
Sep 08 13:28:37.155952 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.155932    2577 manager.go:337] "Starting Device Plugin manager"
Sep 08 13:28:37.156024 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.155972    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 13:28:37.156024 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.155986    2577 server.go:87] "Starting device plugin registration server"
Sep 08 13:28:37.156231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.156220    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 13:28:37.156279 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.156259    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 13:28:37.156401 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.156384    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 13:28:37.156509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.156484    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 13:28:37.156509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.156498    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 13:28:37.156944 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.156920    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 13:28:37.157014 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.156977    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.206099 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.206075    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 13:28:37.207968 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.206125    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 13:28:37.207968 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.206145    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 13:28:37.207968 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.206182    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 13:28:37.208782 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.208764    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:37.257088 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.256986    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.257962 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.257943    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.258061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.257980    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.258061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.257995    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.258061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.258023    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.266461 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.266444    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.266534 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.266465    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-230.ec2.internal\": node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.282671 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.282649    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.307205 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.307184    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"]
Sep 08 13:28:37.307266 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.307244    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.308532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.308518    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.308597 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.308546    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.308597 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.308557    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.309648 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.309635    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.309727 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.309713    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.309769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.309745    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.310306 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310291    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.310398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310295    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.310398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310319    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.310398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310364    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.310398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310378    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.310398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.310382    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.311448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.311432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.311499 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.311464    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:37.312164 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.312138    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:37.312233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.312168    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:37.312233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.312181    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:37.329881 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.329861    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-230.ec2.internal\" not found" node="ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.333331 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.333317    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-230.ec2.internal\" not found" node="ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.382753 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.382727    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.399057 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.399035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.399191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.399061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.399191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.399079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.483476 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.483432    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.499771 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.499848 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499822    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.499886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.499886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.499946 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499882    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/099d282a8bec5dd8dd074b5f725a2d0c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-230.ec2.internal\" (UID: \"099d282a8bec5dd8dd074b5f725a2d0c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.499946 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.499906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8daf278338a7c357076b7909cbd675ea-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal\" (UID: \"8daf278338a7c357076b7909cbd675ea\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.584253 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.584171    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.632654 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.632628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.636767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:37.636279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:37.685189 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.685158    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.785706 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.785672    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.886212 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.886185    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:37.986694 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:37.986655    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.027232 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.027202    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 13:28:38.027662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.027357    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 13:28:38.087715 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.087685    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.134125 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.134093    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:38.134333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.134301    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 13:23:37 +0000 UTC" deadline="2028-06-23 02:30:32.647584716 +0000 UTC"
Sep 08 13:28:38.134426 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.134335    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15685h1m54.513254118s"
Sep 08 13:28:38.135646 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.135626    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:38.135775 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.135660    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:38.135775 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.135673    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:38.188255 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.188116    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.238080 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:38.238049    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8daf278338a7c357076b7909cbd675ea.slice/crio-28cb888ee52002ffbeb855d83e23488fccc2e41bbc8de6a6ecbf4b9a84ba1613 WatchSource:0}: Error finding container 28cb888ee52002ffbeb855d83e23488fccc2e41bbc8de6a6ecbf4b9a84ba1613: Status 404 returned error can't find the container with id 28cb888ee52002ffbeb855d83e23488fccc2e41bbc8de6a6ecbf4b9a84ba1613
Sep 08 13:28:38.242375 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.242336    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:28:38.288837 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.288788    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.296316 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.296295    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:38.323702 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.323674    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:38.389200 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.389169    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.489763 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.489674    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.590555 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:38.590522    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-230.ec2.internal\" not found"
Sep 08 13:28:38.679596 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.679563    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:38.697567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.697538    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:38.707826 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.707799    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:28:38.708927 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.708910    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal"
Sep 08 13:28:38.716313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:38.716295    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:28:39.072930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.072838    2577 apiserver.go:52] "Watching apiserver"
Sep 08 13:28:39.081913 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.081882    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 13:28:39.082253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.082229    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-slzqj","kube-system/konnectivity-agent-xr4gh","kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal","openshift-cluster-node-tuning-operator/tuned-2r456","openshift-dns/node-resolver-z4kt5","openshift-image-registry/node-ca-5dpqb","openshift-multus/network-metrics-daemon-jvplh","openshift-ovn-kubernetes/ovnkube-node-kk9qr","kube-system/global-pull-secret-syncer-pv4qm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-rtj7f","openshift-multus/multus-c7wk8","openshift-network-diagnostics/network-check-target-r2pp5"]
Sep 08 13:28:39.083605 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.083591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.083680 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.083659    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:39.084894 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.084878    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.085838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.085819    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:39.085935 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.085884    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:39.087024 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087000    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.087146 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087097    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 13:28:39.087304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087283    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.087359 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087307    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.087433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087291    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dkk6g\""
Sep 08 13:28:39.087433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.087405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 13:28:39.088569 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.088547    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.089469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.089453    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.089563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.089531    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.089628 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.089579    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.089683 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.089641    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pqgsm\""
Sep 08 13:28:39.090473 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.090455    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.090598 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.090499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-22pgs\""
Sep 08 13:28:39.090598 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.090585    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.090788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.090773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.090842 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.090826    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:39.091494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.091476    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.091603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.091499    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 13:28:39.091603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.091525    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5mhlk\""
Sep 08 13:28:39.092118 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.092101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.092993 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.092974    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.094049 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.094029    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 13:28:39.094222 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.094200    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bfk8r\""
Sep 08 13:28:39.094222 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.094208    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.094404 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.094385    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.094515 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.094498    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.095685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.095661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.096435 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.096412    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 13:28:39.096535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.096456    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 13:28:39.096535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.096500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gqcmx\""
Sep 08 13:28:39.096648 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.096577    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 13:28:39.097411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.097390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.098384 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.097541    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zzt9g\""
Sep 08 13:28:39.098384 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.097694    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 13:28:39.098697 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.098613    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.098697 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.098675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.098697 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.098683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.099202 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.099186    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 13:28:39.099501 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.099481    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 13:28:39.100429 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.100408    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 13:28:39.100541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.100515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 13:28:39.100879 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.100822    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 13:28:39.100946 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.100884    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 13:28:39.101488 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.101468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 13:28:39.101592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.101561    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9qg79\""
Sep 08 13:28:39.101592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.101580    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8mjtt\""
Sep 08 13:28:39.102768 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.102170    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 13:28:39.106688 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.106667    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:28:39.109753 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-netns\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.109855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb16b466-0c5c-4c93-a0ba-d83777864ac7-host\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.109855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-bin\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.109855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-host\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.109855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d577742-adef-4d94-8cd4-fb97c4d709ec-host-slash\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.110070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6qv\" (UniqueName: \"kubernetes.io/projected/2db1f21d-d096-4480-8d48-8749fb44d42c-kube-api-access-nm6qv\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.110070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-socket-dir-parent\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.110070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-netns\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.110070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.109988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-modprobe-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.110070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-tmp\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-socket-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/997febf8-01a1-4706-a5b1-85c0d6253520-ovn-node-metrics-cert\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-systemd\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-run\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-multus-certs\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb16b466-0c5c-4c93-a0ba-d83777864ac7-serviceca\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-ovn\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpghn\" (UniqueName: \"kubernetes.io/projected/4d2cc4c4-102e-4016-9567-27de1398a08a-kube-api-access-vpghn\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.110315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110330    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-konnectivity-ca\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-env-overrides\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-script-lib\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d577742-adef-4d94-8cd4-fb97c4d709ec-iptables-alerter-script\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cni-binary-copy\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-kubelet\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m66n\" (UniqueName: \"kubernetes.io/projected/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-kube-api-access-7m66n\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-slash\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-var-lib-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-netd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-etc-kubernetes\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f28bf03-0013-4c8b-a313-6aef862af55f-tmp-dir\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-systemd-units\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-config\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysconfig\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110931    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-bin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-agent-certs\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.110987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-systemd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-var-lib-kubelet\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-device-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111060    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-system-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-multus\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f28bf03-0013-4c8b-a313-6aef862af55f-hosts-file\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvg7q\" (UniqueName: \"kubernetes.io/projected/997febf8-01a1-4706-a5b1-85c0d6253520-kube-api-access-wvg7q\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111189    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cnibin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-daemon-config\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-log-socket\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-kubernetes\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-conf\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vz4s\" (UniqueName: \"kubernetes.io/projected/882c9adf-dd13-46b6-9471-dbafc81deb3c-kube-api-access-4vz4s\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.111729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111356    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9mbf\" (UniqueName: \"kubernetes.io/projected/8d577742-adef-4d94-8cd4-fb97c4d709ec-kube-api-access-t9mbf\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-conf-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111475    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-tuned\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-dbus\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-etc-selinux\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-k8s-cni-cncf-io\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-node-log\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txdxl\" (UniqueName: \"kubernetes.io/projected/bb16b466-0c5c-4c93-a0ba-d83777864ac7-kube-api-access-txdxl\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111709    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-sys\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-sys-fs\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-hostroot\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.112502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-kubelet\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.113241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-etc-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.113241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-lib-modules\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.113241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-kubelet-config\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.113241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.111991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-registration-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.113241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.112030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9wvc\" (UniqueName: \"kubernetes.io/projected/1f28bf03-0013-4c8b-a313-6aef862af55f-kube-api-access-z9wvc\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.136115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.136090    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ld8vq"
Sep 08 13:28:39.142005 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.141982    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ld8vq"
Sep 08 13:28:39.197944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.197912    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 13:28:39.211080 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.211033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerStarted","Data":"28cb888ee52002ffbeb855d83e23488fccc2e41bbc8de6a6ecbf4b9a84ba1613"}
Sep 08 13:28:39.212335 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-system-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-multus\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f28bf03-0013-4c8b-a313-6aef862af55f-hosts-file\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.212481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.212481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvg7q\" (UniqueName: \"kubernetes.io/projected/997febf8-01a1-4706-a5b1-85c0d6253520-kube-api-access-wvg7q\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.212481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-multus\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-system-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1f28bf03-0013-4c8b-a313-6aef862af55f-hosts-file\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cnibin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-daemon-config\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-log-socket\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.212621    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-kubernetes\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212689    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-log-socket\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212696    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-conf\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz4s\" (UniqueName: \"kubernetes.io/projected/882c9adf-dd13-46b6-9471-dbafc81deb3c-kube-api-access-4vz4s\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-kubernetes\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.212788 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.212764    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:39.712726734 +0000 UTC m=+3.029130273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9mbf\" (UniqueName: \"kubernetes.io/projected/8d577742-adef-4d94-8cd4-fb97c4d709ec-kube-api-access-t9mbf\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-conf-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cnibin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.212981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-conf\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-tuned\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-dbus\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-conf-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-etc-selinux\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-k8s-cni-cncf-io\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-node-log\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.213604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-etc-selinux\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysctl-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txdxl\" (UniqueName: \"kubernetes.io/projected/bb16b466-0c5c-4c93-a0ba-d83777864ac7-kube-api-access-txdxl\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-sys\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-sys-fs\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213293    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213309    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-hostroot\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-kubelet\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-etc-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-lib-modules\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213389    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-node-log\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-kubelet-config\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-registration-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-sys\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.214620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-lib-modules\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-kubelet-config\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-kubelet\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r75tw\" (UniqueName: \"kubernetes.io/projected/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-kube-api-access-r75tw\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9wvc\" (UniqueName: \"kubernetes.io/projected/1f28bf03-0013-4c8b-a313-6aef862af55f-kube-api-access-z9wvc\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-etc-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-sys-fs\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-system-cni-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213692    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-registration-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-k8s-cni-cncf-io\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-hostroot\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213739    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-netns\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-netns\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1933aed9-a5f1-4d70-957b-80c97eddfe96-dbus\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb16b466-0c5c-4c93-a0ba-d83777864ac7-host\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-bin\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb16b466-0c5c-4c93-a0ba-d83777864ac7-host\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-daemon-config\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.215523 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-host\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d577742-adef-4d94-8cd4-fb97c4d709ec-host-slash\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213972    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-bin\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6qv\" (UniqueName: \"kubernetes.io/projected/2db1f21d-d096-4480-8d48-8749fb44d42c-kube-api-access-nm6qv\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-host\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-socket-dir-parent\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.213947    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d577742-adef-4d94-8cd4-fb97c4d709ec-host-slash\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-netns\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-run-netns\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-socket-dir-parent\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-modprobe-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-tmp\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-socket-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-modprobe-d\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/997febf8-01a1-4706-a5b1-85c0d6253520-ovn-node-metrics-cert\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-systemd\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.216474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-run\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-socket-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cnibin\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-multus-certs\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-systemd\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214491    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb16b466-0c5c-4c93-a0ba-d83777864ac7-serviceca\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-run\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-ovn\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214548    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-ovn\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214603    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-run-multus-certs\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpghn\" (UniqueName: \"kubernetes.io/projected/4d2cc4c4-102e-4016-9567-27de1398a08a-kube-api-access-vpghn\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214678    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-konnectivity-ca\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-env-overrides\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-script-lib\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d577742-adef-4d94-8cd4-fb97c4d709ec-iptables-alerter-script\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.214926    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:39.217188 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.214981    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:28:39.714964734 +0000 UTC m=+3.031368267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cni-binary-copy\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.214995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb16b466-0c5c-4c93-a0ba-d83777864ac7-serviceca\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-kubelet\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m66n\" (UniqueName: \"kubernetes.io/projected/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-kube-api-access-7m66n\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-konnectivity-ca\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-script-lib\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-cni-binary-copy\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-kubelet\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215561    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-slash\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-var-lib-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-netd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-cni-netd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8d577742-adef-4d94-8cd4-fb97c4d709ec-iptables-alerter-script\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-var-lib-openvswitch\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-etc-kubernetes\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-host-slash\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f28bf03-0013-4c8b-a313-6aef862af55f-tmp-dir\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.217965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-systemd-units\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-config\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysconfig\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.215964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-env-overrides\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-bin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-etc-kubernetes\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-sysconfig\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-systemd-units\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-multus-cni-dir\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f28bf03-0013-4c8b-a313-6aef862af55f-tmp-dir\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-agent-certs\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-host-var-lib-cni-bin\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-systemd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-var-lib-kubelet\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-device-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2db1f21d-d096-4480-8d48-8749fb44d42c-device-dir\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.218749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d2cc4c4-102e-4016-9567-27de1398a08a-var-lib-kubelet\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.216466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/997febf8-01a1-4706-a5b1-85c0d6253520-run-systemd\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.217246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/997febf8-01a1-4706-a5b1-85c0d6253520-ovnkube-config\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.218938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/997febf8-01a1-4706-a5b1-85c0d6253520-ovn-node-metrics-cert\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.218969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-tmp\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.218992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4d2cc4c4-102e-4016-9567-27de1398a08a-etc-tuned\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.219408 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.219320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/25a6cdf5-7ff9-45f2-9c47-311be2aeeec9-agent-certs\") pod \"konnectivity-agent-xr4gh\" (UID: \"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9\") " pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.224191 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.224064    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:39.224191 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.224089    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:39.224191 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.224103    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:39.224191 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.224165    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:39.724147411 +0000 UTC m=+3.040550944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:39.224668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.224645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvg7q\" (UniqueName: \"kubernetes.io/projected/997febf8-01a1-4706-a5b1-85c0d6253520-kube-api-access-wvg7q\") pod \"ovnkube-node-kk9qr\" (UID: \"997febf8-01a1-4706-a5b1-85c0d6253520\") " pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.225618 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.225571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9mbf\" (UniqueName: \"kubernetes.io/projected/8d577742-adef-4d94-8cd4-fb97c4d709ec-kube-api-access-t9mbf\") pod \"iptables-alerter-slzqj\" (UID: \"8d577742-adef-4d94-8cd4-fb97c4d709ec\") " pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.226128 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.226086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m66n\" (UniqueName: \"kubernetes.io/projected/eea60f6f-0516-43d2-b6c3-6f7c24fbc098-kube-api-access-7m66n\") pod \"multus-c7wk8\" (UID: \"eea60f6f-0516-43d2-b6c3-6f7c24fbc098\") " pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.227211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.227183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9wvc\" (UniqueName: \"kubernetes.io/projected/1f28bf03-0013-4c8b-a313-6aef862af55f-kube-api-access-z9wvc\") pod \"node-resolver-z4kt5\" (UID: \"1f28bf03-0013-4c8b-a313-6aef862af55f\") " pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.227323 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.227209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6qv\" (UniqueName: \"kubernetes.io/projected/2db1f21d-d096-4480-8d48-8749fb44d42c-kube-api-access-nm6qv\") pod \"aws-ebs-csi-driver-node-48jhv\" (UID: \"2db1f21d-d096-4480-8d48-8749fb44d42c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.227682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.227662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txdxl\" (UniqueName: \"kubernetes.io/projected/bb16b466-0c5c-4c93-a0ba-d83777864ac7-kube-api-access-txdxl\") pod \"node-ca-5dpqb\" (UID: \"bb16b466-0c5c-4c93-a0ba-d83777864ac7\") " pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.227825 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.227802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpghn\" (UniqueName: \"kubernetes.io/projected/4d2cc4c4-102e-4016-9567-27de1398a08a-kube-api-access-vpghn\") pod \"tuned-2r456\" (UID: \"4d2cc4c4-102e-4016-9567-27de1398a08a\") " pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.228763 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.228740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz4s\" (UniqueName: \"kubernetes.io/projected/882c9adf-dd13-46b6-9471-dbafc81deb3c-kube-api-access-4vz4s\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.317411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r75tw\" (UniqueName: \"kubernetes.io/projected/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-kube-api-access-r75tw\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-system-cni-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cnibin\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-system-cni-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317926 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317693    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.317926 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.317749    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cnibin\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.318135 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.318112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.318207 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.318183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.318207 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.318197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.326201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.326127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r75tw\" (UniqueName: \"kubernetes.io/projected/6da93b03-2b9d-4eab-a92f-d5ce3f5a8219-kube-api-access-r75tw\") pod \"multus-additional-cni-plugins-rtj7f\" (UID: \"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219\") " pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.397177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.397135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7wk8"
Sep 08 13:28:39.405553 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.405527    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2r456"
Sep 08 13:28:39.414483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.414463    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z4kt5"
Sep 08 13:28:39.418532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.418505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5dpqb"
Sep 08 13:28:39.424454 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.424432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slzqj"
Sep 08 13:28:39.431259 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.431232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:39.442496 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.442473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv"
Sep 08 13:28:39.449387 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.449365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:28:39.455144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.455118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtj7f"
Sep 08 13:28:39.474742 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.474719    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:39.488680 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.488647    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db1f21d_d096_4480_8d48_8749fb44d42c.slice/crio-715552535fcc03d67ea619a377787ce3e4db42338e752e36b369c4f41040d13a WatchSource:0}: Error finding container 715552535fcc03d67ea619a377787ce3e4db42338e752e36b369c4f41040d13a: Status 404 returned error can't find the container with id 715552535fcc03d67ea619a377787ce3e4db42338e752e36b369c4f41040d13a
Sep 08 13:28:39.489671 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.489646    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb16b466_0c5c_4c93_a0ba_d83777864ac7.slice/crio-b1949844dcd4a451c940e9e9e70c9c803e15e3a1161dc6478cddea9dfe026ae8 WatchSource:0}: Error finding container b1949844dcd4a451c940e9e9e70c9c803e15e3a1161dc6478cddea9dfe026ae8: Status 404 returned error can't find the container with id b1949844dcd4a451c940e9e9e70c9c803e15e3a1161dc6478cddea9dfe026ae8
Sep 08 13:28:39.494001 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.493784    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f28bf03_0013_4c8b_a313_6aef862af55f.slice/crio-96bc3c55c29f66fd1f02e4ec19d8ebec96d382d404bc379c283dece965146cbc WatchSource:0}: Error finding container 96bc3c55c29f66fd1f02e4ec19d8ebec96d382d404bc379c283dece965146cbc: Status 404 returned error can't find the container with id 96bc3c55c29f66fd1f02e4ec19d8ebec96d382d404bc379c283dece965146cbc
Sep 08 13:28:39.496339 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.496314    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997febf8_01a1_4706_a5b1_85c0d6253520.slice/crio-fc72f4ed7d652943cf4223929944c69c17f54695fa09a559ad5e9e5ba3f1fa1b WatchSource:0}: Error finding container fc72f4ed7d652943cf4223929944c69c17f54695fa09a559ad5e9e5ba3f1fa1b: Status 404 returned error can't find the container with id fc72f4ed7d652943cf4223929944c69c17f54695fa09a559ad5e9e5ba3f1fa1b
Sep 08 13:28:39.497149 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.497078    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d577742_adef_4d94_8cd4_fb97c4d709ec.slice/crio-01e011b255b6bb4f4714bc8f7d417ed170f6cb328014456fbaac6f08cf852949 WatchSource:0}: Error finding container 01e011b255b6bb4f4714bc8f7d417ed170f6cb328014456fbaac6f08cf852949: Status 404 returned error can't find the container with id 01e011b255b6bb4f4714bc8f7d417ed170f6cb328014456fbaac6f08cf852949
Sep 08 13:28:39.498575 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.498090    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d2cc4c4_102e_4016_9567_27de1398a08a.slice/crio-1c872195581271d215cd47457c39d798c4d7253d1225cc51379e23dda15f782a WatchSource:0}: Error finding container 1c872195581271d215cd47457c39d798c4d7253d1225cc51379e23dda15f782a: Status 404 returned error can't find the container with id 1c872195581271d215cd47457c39d798c4d7253d1225cc51379e23dda15f782a
Sep 08 13:28:39.499479 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.499320    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da93b03_2b9d_4eab_a92f_d5ce3f5a8219.slice/crio-658236ea490e73aa24eeaccd22ea8debe1c975052b22c38e579b7c8f481d3942 WatchSource:0}: Error finding container 658236ea490e73aa24eeaccd22ea8debe1c975052b22c38e579b7c8f481d3942: Status 404 returned error can't find the container with id 658236ea490e73aa24eeaccd22ea8debe1c975052b22c38e579b7c8f481d3942
Sep 08 13:28:39.502259 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.502233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea60f6f_0516_43d2_b6c3_6f7c24fbc098.slice/crio-92d44476b220a4515681b2aeef29d521a073bf14c0e422d88cf63d3c4866f1d6 WatchSource:0}: Error finding container 92d44476b220a4515681b2aeef29d521a073bf14c0e422d88cf63d3c4866f1d6: Status 404 returned error can't find the container with id 92d44476b220a4515681b2aeef29d521a073bf14c0e422d88cf63d3c4866f1d6
Sep 08 13:28:39.502623 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:28:39.502598    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a6cdf5_7ff9_45f2_9c47_311be2aeeec9.slice/crio-e7cf4933fbffe909f225249f595629f8e0fbea42b378d2532aa6f4a641b41223 WatchSource:0}: Error finding container e7cf4933fbffe909f225249f595629f8e0fbea42b378d2532aa6f4a641b41223: Status 404 returned error can't find the container with id e7cf4933fbffe909f225249f595629f8e0fbea42b378d2532aa6f4a641b41223
Sep 08 13:28:39.721170 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.721130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:39.721341 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.721191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:39.721341 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.721295    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:39.721500 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.721295    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:39.721500 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.721397    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:28:40.721375274 +0000 UTC m=+4.037778822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:39.721500 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.721417    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:40.721408609 +0000 UTC m=+4.037812141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:39.822514 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:39.822484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:39.822676 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.822655    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:39.822736 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.822684    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:39.822736 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.822695    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:39.822801 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:39.822747    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:40.822732511 +0000 UTC m=+4.139136045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:40.144067 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.143959    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:23:39 +0000 UTC" deadline="2028-03-20 15:58:50.857628532 +0000 UTC"
Sep 08 13:28:40.144067 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.144006    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13418h30m10.713627925s"
Sep 08 13:28:40.207481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.206949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:40.207481 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.207089    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:40.219314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.219119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"fc72f4ed7d652943cf4223929944c69c17f54695fa09a559ad5e9e5ba3f1fa1b"}
Sep 08 13:28:40.227826 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.227792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slzqj" event={"ID":"8d577742-adef-4d94-8cd4-fb97c4d709ec","Type":"ContainerStarted","Data":"01e011b255b6bb4f4714bc8f7d417ed170f6cb328014456fbaac6f08cf852949"}
Sep 08 13:28:40.235800 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.235733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xr4gh" event={"ID":"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9","Type":"ContainerStarted","Data":"e7cf4933fbffe909f225249f595629f8e0fbea42b378d2532aa6f4a641b41223"}
Sep 08 13:28:40.238206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.238119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerStarted","Data":"658236ea490e73aa24eeaccd22ea8debe1c975052b22c38e579b7c8f481d3942"}
Sep 08 13:28:40.241478 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.241350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2r456" event={"ID":"4d2cc4c4-102e-4016-9567-27de1398a08a","Type":"ContainerStarted","Data":"1c872195581271d215cd47457c39d798c4d7253d1225cc51379e23dda15f782a"}
Sep 08 13:28:40.245068 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.244989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4kt5" event={"ID":"1f28bf03-0013-4c8b-a313-6aef862af55f","Type":"ContainerStarted","Data":"96bc3c55c29f66fd1f02e4ec19d8ebec96d382d404bc379c283dece965146cbc"}
Sep 08 13:28:40.247514 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.247449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dpqb" event={"ID":"bb16b466-0c5c-4c93-a0ba-d83777864ac7","Type":"ContainerStarted","Data":"b1949844dcd4a451c940e9e9e70c9c803e15e3a1161dc6478cddea9dfe026ae8"}
Sep 08 13:28:40.252980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.252894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv" event={"ID":"2db1f21d-d096-4480-8d48-8749fb44d42c","Type":"ContainerStarted","Data":"715552535fcc03d67ea619a377787ce3e4db42338e752e36b369c4f41040d13a"}
Sep 08 13:28:40.257469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.257442    2577 generic.go:356] "Generic (PLEG): container finished" podID="8daf278338a7c357076b7909cbd675ea" containerID="49e8ca981fb3726714730a06b737c620e30857c078dc7d1678c95732d02e9a2e" exitCode=0
Sep 08 13:28:40.257652 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.257628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerDied","Data":"49e8ca981fb3726714730a06b737c620e30857c078dc7d1678c95732d02e9a2e"}
Sep 08 13:28:40.260835 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.260810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7wk8" event={"ID":"eea60f6f-0516-43d2-b6c3-6f7c24fbc098","Type":"ContainerStarted","Data":"92d44476b220a4515681b2aeef29d521a073bf14c0e422d88cf63d3c4866f1d6"}
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.731578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.731691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.731825    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.731892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.731871903 +0000 UTC m=+6.048275458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.732299    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:40.732447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.732349    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.732333619 +0000 UTC m=+6.048737158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:40.832733 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:40.832687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:40.832952 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.832933    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:40.833015 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.832965    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:40.833015 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.832978    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:40.833125 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:40.833040    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.83302046 +0000 UTC m=+6.149423999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:41.144806 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.144734    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:23:39 +0000 UTC" deadline="2028-02-24 01:05:21.932287142 +0000 UTC"
Sep 08 13:28:41.144806 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.144803    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12803h36m40.787489326s"
Sep 08 13:28:41.209119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.209043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:41.209302 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:41.209191    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:41.209646 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.209623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:41.209835 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:41.209733    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:41.271333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.270596    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" event={"ID":"8daf278338a7c357076b7909cbd675ea","Type":"ContainerStarted","Data":"6addc5c084e58d5b28016860daecced7860a871463590289b06077ba1e3eb071"}
Sep 08 13:28:41.287449 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:41.287142    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-230.ec2.internal" podStartSLOduration=3.287122062 podStartE2EDuration="3.287122062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:41.286124077 +0000 UTC m=+4.602527638" watchObservedRunningTime="2026-09-08 13:28:41.287122062 +0000 UTC m=+4.603525620"
Sep 08 13:28:42.206750 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:42.206717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:42.207256 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.206832    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:42.749963 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:42.749909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:42.750154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:42.750017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:42.750211 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.750164    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:42.750276 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.750256    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:28:46.750237907 +0000 UTC m=+10.066641445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:42.750731 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.750714    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:42.750803 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.750769    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:46.75075355 +0000 UTC m=+10.067157081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:42.851327 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:42.851286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:42.851491 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.851459    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:42.851491 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.851478    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:42.851491 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.851488    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:42.851591 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:42.851543    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:46.851525632 +0000 UTC m=+10.167929166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:43.207484 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:43.207451    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:43.207845 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:43.207594    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:43.208021 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:43.208001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:43.208107 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:43.208092    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:44.207070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:44.206482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:44.207070 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:44.206609    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:45.208755 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:45.208715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:45.209196 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:45.208880    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:45.209290 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:45.209270    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:45.209403 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:45.209383    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:46.206753 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:46.206524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:46.206939 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.206854    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:46.780186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:46.780252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.780392    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.780429    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.780452    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:54.780433224 +0000 UTC m=+18.096836755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:46.780785 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.780470    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:28:54.780459193 +0000 UTC m=+18.096862744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:46.881038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:46.881000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:46.881225 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.881211    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:46.881291 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.881232    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:46.881291 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.881244    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:46.881408 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:46.881302    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:54.881283194 +0000 UTC m=+18.197686757 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:47.208608 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:47.208256    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:47.208608 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:47.208400    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:47.208608 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:47.208457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:47.208608 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:47.208571    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:48.206450 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:48.206420    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:48.206786 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:48.206533    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:49.207162 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:49.207122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:49.207162 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:49.207153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:49.207683 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:49.207246    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:49.207683 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:49.207609    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:50.206502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:50.206466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:50.206679 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:50.206598    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:51.206819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:51.206789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:51.207281 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:51.206789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:51.207281 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:51.206919    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:51.207281 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:51.206989    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:52.206545 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:52.206511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:52.206702 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:52.206617    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:53.206877 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:53.206845    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:53.207370 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:53.206976    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:53.207370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:53.207025    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:53.207370 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:53.207107    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:54.206930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:54.206896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:54.207443 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.207002    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:54.837181 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:54.837147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:54.837390 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:54.837197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:54.837390 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.837371    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:54.837390 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.837372    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:54.837545 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.837439    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:29:10.837422864 +0000 UTC m=+34.153826397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:54.837545 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.837456    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:10.837450096 +0000 UTC m=+34.153853627 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:54.938316 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:54.938277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:54.938495 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.938472    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:54.938495 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.938496    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:54.938620 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.938510    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:54.938620 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:54.938575    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:10.938555353 +0000 UTC m=+34.254958902 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:55.206600 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:55.206561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:55.206783 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:55.206615    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:55.206783 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:55.206707    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:55.206889 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:55.206803    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:56.207047 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:56.207009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:56.207512 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:56.207200    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:57.208105 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.207879    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:57.208943 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:57.208215    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:57.208943 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.207961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:57.208943 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:57.208314    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:57.306438 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.306399    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" event={"ID":"099d282a8bec5dd8dd074b5f725a2d0c","Type":"ContainerStarted","Data":"9d0f15f12420f79d76909ad494761e33d1b9421343f8068fa81b5ed40b404ebd"}
Sep 08 13:28:57.308542 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.308443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7wk8" event={"ID":"eea60f6f-0516-43d2-b6c3-6f7c24fbc098","Type":"ContainerStarted","Data":"a830d5f2238ae91763a6280e07c8833945e0b1f37433090de6469ba8f2c7beef"}
Sep 08 13:28:57.315144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315086    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"e9ed0aa21edfcb9a1a7a7db91edd89d0bd2b96a455eec36f4a7b23dcc6f67a3b"}
Sep 08 13:28:57.315283 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"cf469bbae0fd4a32085c590ae0dd77ac3b27df1af9a11660ac68b442167a8cb3"}
Sep 08 13:28:57.315283 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"c1498d30a3e881309b184002b366eb3dbc0843853b8574596df5828c242d17eb"}
Sep 08 13:28:57.315497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"60a6e0f8efad4ca9dc1760c839158fb744f51d1b69b993ceabb335b505e7f7fb"}
Sep 08 13:28:57.315497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"9d7c49256f9e52a7f8f35d344a20a1edb83dfe8bce271738c8802cbe94e6c4c2"}
Sep 08 13:28:57.315497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.315395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"e7ae76088367123b2b858529dfe1a64f72ea97ac35f38c4b5f292747794d2a0e"}
Sep 08 13:28:57.318323 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.318296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xr4gh" event={"ID":"25a6cdf5-7ff9-45f2-9c47-311be2aeeec9","Type":"ContainerStarted","Data":"d66b043022fbc430005c9ff4de44fccdc07324773be9079ea8ab9b6f17ec8024"}
Sep 08 13:28:57.321421 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.321393    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="0aa397295babde399fa24bb9f08c0fbcb4287e8111c0783a70a00a346f5158b1" exitCode=0
Sep 08 13:28:57.321546 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.321482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"0aa397295babde399fa24bb9f08c0fbcb4287e8111c0783a70a00a346f5158b1"}
Sep 08 13:28:57.324774 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.324751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2r456" event={"ID":"4d2cc4c4-102e-4016-9567-27de1398a08a","Type":"ContainerStarted","Data":"144a15e6252505dc3f0e0f5a730118a641dac2d9a4f71feeb2da5298af0adce2"}
Sep 08 13:28:57.325712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.325645    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c7wk8" podStartSLOduration=3.198632949 podStartE2EDuration="20.325626579s" totalImagesPullingTime="17.12699363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.325234059 +0000 UTC m=+20.641637629" watchObservedRunningTime="2026-09-08 13:28:57.325626579 +0000 UTC m=+20.642030132"
Sep 08 13:28:57.331538 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.331502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z4kt5" event={"ID":"1f28bf03-0013-4c8b-a313-6aef862af55f","Type":"ContainerStarted","Data":"9c61566a2fcbccb77309727070180266a1c85f7a8aecffe9989ac770eee7ae1f"}
Sep 08 13:28:57.335049 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.335019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5dpqb" event={"ID":"bb16b466-0c5c-4c93-a0ba-d83777864ac7","Type":"ContainerStarted","Data":"18dec92891a5e0a675e6500f8cc7249da5a2308a5d7f26a77c8a10b87ba45e74"}
Sep 08 13:28:57.338025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.337996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv" event={"ID":"2db1f21d-d096-4480-8d48-8749fb44d42c","Type":"ContainerStarted","Data":"e6b3f50d9c0a95acacd9f34bb3a1e8643edee57892c491b35f7662e9c2598e97"}
Sep 08 13:28:57.356796 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.356730    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xr4gh" podStartSLOduration=7.190011087 podStartE2EDuration="20.356708768s" totalImagesPullingTime="13.166697681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.356633226 +0000 UTC m=+20.673036778" watchObservedRunningTime="2026-09-08 13:28:57.356708768 +0000 UTC m=+20.673112321"
Sep 08 13:28:57.356970 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.356877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2r456" podStartSLOduration=3.301969826 podStartE2EDuration="20.356868053s" totalImagesPullingTime="17.054898227s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.34307601 +0000 UTC m=+20.659479581" watchObservedRunningTime="2026-09-08 13:28:57.356868053 +0000 UTC m=+20.673271607"
Sep 08 13:28:57.409279 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:57.409213    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z4kt5" podStartSLOduration=3.350675674 podStartE2EDuration="20.409187297s" totalImagesPullingTime="17.058511623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.393004647 +0000 UTC m=+20.709408217" watchObservedRunningTime="2026-09-08 13:28:57.409187297 +0000 UTC m=+20.725590851"
Sep 08 13:28:58.207368 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.207319    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:28:58.207599 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:58.207560    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:28:58.343268 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.343097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slzqj" event={"ID":"8d577742-adef-4d94-8cd4-fb97c4d709ec","Type":"ContainerStarted","Data":"59a8ffd6a41c7d1e9659e57f64c2b7099207324b71243015f7fff2642ab259e4"}
Sep 08 13:28:58.355998 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.355929    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-slzqj" podStartSLOduration=4.284778653 podStartE2EDuration="21.355906071s" totalImagesPullingTime="17.071127418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:58.355376067 +0000 UTC m=+21.671779645" watchObservedRunningTime="2026-09-08 13:28:58.355906071 +0000 UTC m=+21.672309626"
Sep 08 13:28:58.356325 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.356282    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5dpqb" podStartSLOduration=8.177142207 podStartE2EDuration="21.356266949s" totalImagesPullingTime="13.179124742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.410918541 +0000 UTC m=+20.727322110" watchObservedRunningTime="2026-09-08 13:28:58.356266949 +0000 UTC m=+21.672670501"
Sep 08 13:28:58.488163 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.488123    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 13:28:58.644973 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.644936    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:58.645603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:58.645581    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:28:59.166895 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.166799    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T13:28:58.488145613Z","UUID":"7e9c4094-7bee-4007-964d-285330065e6a","Handler":null,"Name":"","Endpoint":""}
Sep 08 13:28:59.168404 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.168385    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 13:28:59.168528 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.168417    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 13:28:59.206629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.206602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:28:59.206629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.206622    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:28:59.206863 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:59.206736    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:28:59.206863 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:28:59.206848    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:28:59.348207 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.348173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" event={"ID":"099d282a8bec5dd8dd074b5f725a2d0c","Type":"ContainerStarted","Data":"64d9747aa3801cb19f9e8ffa3b787fee7198d96a3ebb4a4b58cd4cb6b33f2d86"}
Sep 08 13:28:59.360376 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.360320    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv" event={"ID":"2db1f21d-d096-4480-8d48-8749fb44d42c","Type":"ContainerStarted","Data":"e107d0b5c6b66dfea08a584352d8b45ffedb99813ca6ad54d1134c36d03e5f49"}
Sep 08 13:28:59.370159 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:28:59.370100    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-230.ec2.internal" podStartSLOduration=21.37007918 podStartE2EDuration="21.37007918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:59.369865819 +0000 UTC m=+22.686269373" watchObservedRunningTime="2026-09-08 13:28:59.37007918 +0000 UTC m=+22.686482735"
Sep 08 13:29:00.207310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:00.207273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:00.207594 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:00.207447    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:00.366280 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:00.366243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"4ce55782caa781ca27fa4ac0eafb6b25248bb1a7b3dcc1578976bab893f8d6dc"}
Sep 08 13:29:00.369439 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:00.369400    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv" event={"ID":"2db1f21d-d096-4480-8d48-8749fb44d42c","Type":"ContainerStarted","Data":"2e4d93cdcfbe23c0bec7d63b055374f0dff7daee186b8e73909722f404bed839"}
Sep 08 13:29:00.369565 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:00.369534    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 13:29:00.385873 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:00.385826    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-48jhv" podStartSLOduration=3.886548849 podStartE2EDuration="23.385808782s" totalImagesPullingTime="19.499259933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:00.385262841 +0000 UTC m=+23.701666414" watchObservedRunningTime="2026-09-08 13:29:00.385808782 +0000 UTC m=+23.702212336"
Sep 08 13:29:01.207172 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:01.207141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:01.207390 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:01.207141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:01.207390 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:01.207291    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:01.207517 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:01.207384    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:02.207154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.206984    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:02.207813 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:02.207240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:02.376845 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.376761    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" event={"ID":"997febf8-01a1-4706-a5b1-85c0d6253520","Type":"ContainerStarted","Data":"6f585b715837a3e70af6de708cfb792cd4d6a3d0931af178f4599526549c9eac"}
Sep 08 13:29:02.377030 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.377013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:02.378684 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.378651    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="ffb37a9adeca18b9cf9d23c4d6668d1c82043af0742246e36bb2330dd5c3174f" exitCode=0
Sep 08 13:29:02.378783 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.378699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"ffb37a9adeca18b9cf9d23c4d6668d1c82043af0742246e36bb2330dd5c3174f"}
Sep 08 13:29:02.393290 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.393266    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:02.403386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:02.403329    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr" podStartSLOduration=8.320492034 podStartE2EDuration="25.403316866s" totalImagesPullingTime="17.082824832s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:02.402653268 +0000 UTC m=+25.719056836" watchObservedRunningTime="2026-09-08 13:29:02.403316866 +0000 UTC m=+25.719720418"
Sep 08 13:29:03.207233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.207198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:03.207233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.207235    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:03.207750 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:03.207390    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:03.207750 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:03.207417    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:03.385125 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.385086    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="a6ac5ea596e1352a9ebe2f6f80b677614ded4bbb9f983a0c9894382ef90c5ab1" exitCode=0
Sep 08 13:29:03.385304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.385182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"a6ac5ea596e1352a9ebe2f6f80b677614ded4bbb9f983a0c9894382ef90c5ab1"}
Sep 08 13:29:03.385801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.385776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:03.385932 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.385852    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:03.402135 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.402110    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:03.548958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.548787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:29:03.549137 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.549059    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 13:29:03.549521 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.549492    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xr4gh"
Sep 08 13:29:03.701619 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.701591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pv4qm"]
Sep 08 13:29:03.701814 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.701720    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:03.701881 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:03.701840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:03.702315 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.702295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jvplh"]
Sep 08 13:29:03.702417 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.702394    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:03.702486 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:03.702468    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:03.704293 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.704261    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2pp5"]
Sep 08 13:29:03.704422 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:03.704367    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:03.704475 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:03.704436    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:04.389858 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:04.389822    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="c63e8a7ecf3ee67eda08bb072c1357d9265526bb2492847c02fc3bb2761e6efa" exitCode=0
Sep 08 13:29:04.390380 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:04.389920    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"c63e8a7ecf3ee67eda08bb072c1357d9265526bb2492847c02fc3bb2761e6efa"}
Sep 08 13:29:05.207386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:05.207333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:05.207575 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:05.207361    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:05.207575 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:05.207495    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:05.207575 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:05.207511    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:05.207748 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:05.207601    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:05.207748 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:05.207691    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:07.208029 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:07.207993    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:07.208447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:07.208112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:07.208447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:07.208145    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:07.208447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:07.208164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:07.208447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:07.208257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:07.208447 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:07.208361    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:09.206827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.206801    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:09.207108 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.206797    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:09.207108 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:09.206918    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jvplh" podUID="882c9adf-dd13-46b6-9471-dbafc81deb3c"
Sep 08 13:29:09.207108 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:09.206967    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r2pp5" podUID="7c5b5fd3-f594-40eb-8301-548dd9f6b468"
Sep 08 13:29:09.207108 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.206808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:09.207108 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:09.207045    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pv4qm" podUID="1933aed9-a5f1-4d70-957b-80c97eddfe96"
Sep 08 13:29:09.407133 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.406913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerStarted","Data":"af6927b127aca151b3678e58d4ca810aa8cd0c0699b25ae8f7a34c0a0041a293"}
Sep 08 13:29:09.495641 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.495604    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-230.ec2.internal" event="NodeReady"
Sep 08 13:29:09.495734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.495729    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 13:29:09.536058 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.536031    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nn66g"]
Sep 08 13:29:09.554515 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.554490    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n5nwm"]
Sep 08 13:29:09.554652 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.554602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.556819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.556787    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 13:29:09.556923 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.556818    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 13:29:09.556923 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.556795    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwgzv\""
Sep 08 13:29:09.565005 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.564985    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn66g"]
Sep 08 13:29:09.565090 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.565020    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5nwm"]
Sep 08 13:29:09.565122 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.565092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.566939 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.566911    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 13:29:09.567032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.566950    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rbspx\""
Sep 08 13:29:09.567032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.566988    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 13:29:09.567032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.566914    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 13:29:09.650118 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07486108-6f0e-4822-a7bd-aedfc71b4112-cert\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.650265 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b7648c-e184-4582-a115-a72957ac4884-tmp-dir\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.650265 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98b7648c-e184-4582-a115-a72957ac4884-config-volume\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.650265 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.650398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzkwk\" (UniqueName: \"kubernetes.io/projected/98b7648c-e184-4582-a115-a72957ac4884-kube-api-access-kzkwk\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.650398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.650301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqprw\" (UniqueName: \"kubernetes.io/projected/07486108-6f0e-4822-a7bd-aedfc71b4112-kube-api-access-sqprw\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.751358 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzkwk\" (UniqueName: \"kubernetes.io/projected/98b7648c-e184-4582-a115-a72957ac4884-kube-api-access-kzkwk\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.751486 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqprw\" (UniqueName: \"kubernetes.io/projected/07486108-6f0e-4822-a7bd-aedfc71b4112-kube-api-access-sqprw\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.751486 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07486108-6f0e-4822-a7bd-aedfc71b4112-cert\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.751574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b7648c-e184-4582-a115-a72957ac4884-tmp-dir\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.751574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98b7648c-e184-4582-a115-a72957ac4884-config-volume\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.751574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.751707 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:09.751642    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:09.751707 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:09.751701    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:10.251682705 +0000 UTC m=+33.568086235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:09.751906 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.751881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98b7648c-e184-4582-a115-a72957ac4884-tmp-dir\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.752091 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.752071    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/98b7648c-e184-4582-a115-a72957ac4884-config-volume\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.755616 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.755601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07486108-6f0e-4822-a7bd-aedfc71b4112-cert\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.759821 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.759799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqprw\" (UniqueName: \"kubernetes.io/projected/07486108-6f0e-4822-a7bd-aedfc71b4112-kube-api-access-sqprw\") pod \"ingress-canary-n5nwm\" (UID: \"07486108-6f0e-4822-a7bd-aedfc71b4112\") " pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:09.759821 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.759815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzkwk\" (UniqueName: \"kubernetes.io/projected/98b7648c-e184-4582-a115-a72957ac4884-kube-api-access-kzkwk\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:09.873196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:09.873161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n5nwm"
Sep 08 13:29:10.184165 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.183775    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n5nwm"]
Sep 08 13:29:10.187744 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:29:10.187721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07486108_6f0e_4822_a7bd_aedfc71b4112.slice/crio-40ffdac6bab7be9e314699a22e76cf5f2ee3d790e5012e990095f605a4685eee WatchSource:0}: Error finding container 40ffdac6bab7be9e314699a22e76cf5f2ee3d790e5012e990095f605a4685eee: Status 404 returned error can't find the container with id 40ffdac6bab7be9e314699a22e76cf5f2ee3d790e5012e990095f605a4685eee
Sep 08 13:29:10.255170 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.255139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:10.255731 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.255274    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:10.255731 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.255334    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:11.255318599 +0000 UTC m=+34.571722130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:10.410781 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.410744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5nwm" event={"ID":"07486108-6f0e-4822-a7bd-aedfc71b4112","Type":"ContainerStarted","Data":"40ffdac6bab7be9e314699a22e76cf5f2ee3d790e5012e990095f605a4685eee"}
Sep 08 13:29:10.413665 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.413642    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="af6927b127aca151b3678e58d4ca810aa8cd0c0699b25ae8f7a34c0a0041a293" exitCode=0
Sep 08 13:29:10.413779 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.413676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"af6927b127aca151b3678e58d4ca810aa8cd0c0699b25ae8f7a34c0a0041a293"}
Sep 08 13:29:10.860400 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.860299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:10.860400 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.860393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:10.860664 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.860477    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:29:10.860664 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.860541    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:29:10.860664 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.860558    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:29:42.860539353 +0000 UTC m=+66.176942906 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:29:10.860664 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.860602    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret podName:1933aed9-a5f1-4d70-957b-80c97eddfe96 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:42.860584569 +0000 UTC m=+66.176988100 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret") pod "global-pull-secret-syncer-pv4qm" (UID: "1933aed9-a5f1-4d70-957b-80c97eddfe96") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:29:10.961153 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:10.961119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:10.961360 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.961316    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:29:10.961435 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.961369    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:29:10.961435 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.961384    2577 projected.go:196] Error preparing data for projected volume kube-api-access-g8dzx for pod openshift-network-diagnostics/network-check-target-r2pp5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:29:10.961544 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:10.961449    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx podName:7c5b5fd3-f594-40eb-8301-548dd9f6b468 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:42.961431467 +0000 UTC m=+66.277835011 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-g8dzx" (UniqueName: "kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx") pod "network-check-target-r2pp5" (UID: "7c5b5fd3-f594-40eb-8301-548dd9f6b468") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:29:11.206524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.206488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:11.206683 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.206492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:11.206683 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.206637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:11.209044 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209023    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:29:11.209818 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209576    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n94wj\""
Sep 08 13:29:11.209818 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209590    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:29:11.209818 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:29:11.209818 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209628    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:29:11.209818 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.209704    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56h6n\""
Sep 08 13:29:11.263046 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.263022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:11.263551 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:11.263149    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:11.263551 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:11.263208    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:13.263189264 +0000 UTC m=+36.579592805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:11.419894 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.419859    2577 generic.go:356] "Generic (PLEG): container finished" podID="6da93b03-2b9d-4eab-a92f-d5ce3f5a8219" containerID="5a5ab9e4778fecfa52dd4819c9ffc3b3ba749975f565f45439f905865118a531" exitCode=0
Sep 08 13:29:11.420037 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:11.419929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerDied","Data":"5a5ab9e4778fecfa52dd4819c9ffc3b3ba749975f565f45439f905865118a531"}
Sep 08 13:29:12.426113 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:12.425950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" event={"ID":"6da93b03-2b9d-4eab-a92f-d5ce3f5a8219","Type":"ContainerStarted","Data":"123f514aba0c9bfa67255711916817a0fd6657517455c6dd41198534c6bb662b"}
Sep 08 13:29:12.445304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:12.445256    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rtj7f" podStartSLOduration=7.655897199 podStartE2EDuration="35.445240369s" totalImagesPullingTime="27.78934317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:12.444286715 +0000 UTC m=+35.760690267" watchObservedRunningTime="2026-09-08 13:29:12.445240369 +0000 UTC m=+35.761643923"
Sep 08 13:29:13.277736 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:13.277700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:13.277914 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:13.277823    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:13.277914 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:13.277873    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:17.277859574 +0000 UTC m=+40.594263105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:13.430015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:13.429981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n5nwm" event={"ID":"07486108-6f0e-4822-a7bd-aedfc71b4112","Type":"ContainerStarted","Data":"128168863fa9664e112f5234ae92c1fa65e28f350a2fa0e292d90640dea81652"}
Sep 08 13:29:13.449428 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:13.449386    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n5nwm" podStartSLOduration=2.119843477 podStartE2EDuration="4.449370494s" totalImagesPullingTime="2.329527017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:13.449085851 +0000 UTC m=+36.765489422" watchObservedRunningTime="2026-09-08 13:29:13.449370494 +0000 UTC m=+36.765774047"
Sep 08 13:29:17.305705 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:17.305665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:17.306118 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:17.305815    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:17.306118 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:17.305885    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:25.30586531 +0000 UTC m=+48.622268840 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:25.362746 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:25.362707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:25.363153 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:25.362839    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:25.363153 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:25.362892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:41.36287703 +0000 UTC m=+64.679280561 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:35.403187 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:35.403161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kk9qr"
Sep 08 13:29:41.377680 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:41.377626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:29:41.378146 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:41.377769    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:41.378146 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:41.377830    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:13.377814375 +0000 UTC m=+96.694217910 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:42.887805 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.887767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:42.888191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.887833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:29:42.890357 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.890325    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:29:42.890462 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.890430    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:29:42.898741 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:42.898722    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 13:29:42.898828 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:29:42.898788    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs podName:882c9adf-dd13-46b6-9471-dbafc81deb3c nodeName:}" failed. No retries permitted until 2026-09-08 13:30:46.898773479 +0000 UTC m=+130.215177010 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs") pod "network-metrics-daemon-jvplh" (UID: "882c9adf-dd13-46b6-9471-dbafc81deb3c") : secret "metrics-daemon-secret" not found
Sep 08 13:29:42.901850 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.901819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1933aed9-a5f1-4d70-957b-80c97eddfe96-original-pull-secret\") pod \"global-pull-secret-syncer-pv4qm\" (UID: \"1933aed9-a5f1-4d70-957b-80c97eddfe96\") " pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:42.988799 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.988772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:42.991032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:42.991016    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:29:43.001658 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.001641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:29:43.012795 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.012769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8dzx\" (UniqueName: \"kubernetes.io/projected/7c5b5fd3-f594-40eb-8301-548dd9f6b468-kube-api-access-g8dzx\") pod \"network-check-target-r2pp5\" (UID: \"7c5b5fd3-f594-40eb-8301-548dd9f6b468\") " pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:43.025475 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.025458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pv4qm"
Sep 08 13:29:43.033069 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.033051    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n94wj\""
Sep 08 13:29:43.041482 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.041465    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:43.293470 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.293395    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pv4qm"]
Sep 08 13:29:43.296835 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:29:43.296801    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1933aed9_a5f1_4d70_957b_80c97eddfe96.slice/crio-ef996b033762c8cc5f655ae251d4c8d402a10033514c398e0192b83c4740f88a WatchSource:0}: Error finding container ef996b033762c8cc5f655ae251d4c8d402a10033514c398e0192b83c4740f88a: Status 404 returned error can't find the container with id ef996b033762c8cc5f655ae251d4c8d402a10033514c398e0192b83c4740f88a
Sep 08 13:29:43.315543 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.315521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r2pp5"]
Sep 08 13:29:43.318243 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:29:43.318223    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5b5fd3_f594_40eb_8301_548dd9f6b468.slice/crio-cbff831cc065c18d42f5dadb2d777e18397ba980ccceffa6be3a5f0451e8d375 WatchSource:0}: Error finding container cbff831cc065c18d42f5dadb2d777e18397ba980ccceffa6be3a5f0451e8d375: Status 404 returned error can't find the container with id cbff831cc065c18d42f5dadb2d777e18397ba980ccceffa6be3a5f0451e8d375
Sep 08 13:29:43.498644 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.498606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pv4qm" event={"ID":"1933aed9-a5f1-4d70-957b-80c97eddfe96","Type":"ContainerStarted","Data":"ef996b033762c8cc5f655ae251d4c8d402a10033514c398e0192b83c4740f88a"}
Sep 08 13:29:43.499730 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:43.499706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2pp5" event={"ID":"7c5b5fd3-f594-40eb-8301-548dd9f6b468","Type":"ContainerStarted","Data":"cbff831cc065c18d42f5dadb2d777e18397ba980ccceffa6be3a5f0451e8d375"}
Sep 08 13:29:47.514837 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:47.514797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pv4qm" event={"ID":"1933aed9-a5f1-4d70-957b-80c97eddfe96","Type":"ContainerStarted","Data":"1ab32731e27f11794a81bf4ed4b41924d714c67f46ab1db39bbd7d440927925f"}
Sep 08 13:29:47.530143 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:47.530094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pv4qm" podStartSLOduration=66.855339421 podStartE2EDuration="1m10.530076741s" totalImagesPullingTime="3.67473732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:47.530042203 +0000 UTC m=+70.846445772" watchObservedRunningTime="2026-09-08 13:29:47.530076741 +0000 UTC m=+70.846480285"
Sep 08 13:29:52.528968 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:52.528879    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r2pp5" event={"ID":"7c5b5fd3-f594-40eb-8301-548dd9f6b468","Type":"ContainerStarted","Data":"1420f68f72344a6a47b58e737107d5b081c2bc0694a933ef783553fb7810339b"}
Sep 08 13:29:52.529312 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:52.529037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:29:52.543660 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:29:52.543617    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r2pp5" podStartSLOduration=66.640895054 podStartE2EDuration="1m15.543603317s" totalImagesPullingTime="8.902708263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:52.542655578 +0000 UTC m=+75.859059132" watchObservedRunningTime="2026-09-08 13:29:52.543603317 +0000 UTC m=+75.860006869"
Sep 08 13:30:13.379357 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:13.379312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:30:13.379753 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:13.379447    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:30:13.379753 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:13.379519    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls podName:98b7648c-e184-4582-a115-a72957ac4884 nodeName:}" failed. No retries permitted until 2026-09-08 13:31:17.379504601 +0000 UTC m=+160.695908133 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls") pod "dns-default-nn66g" (UID: "98b7648c-e184-4582-a115-a72957ac4884") : secret "dns-default-metrics-tls" not found
Sep 08 13:30:21.301815 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.301780    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"]
Sep 08 13:30:21.304146 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.304126    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"
Sep 08 13:30:21.306436 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.306416    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.307622 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.307590    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.307622 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.307594    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d6sd6\""
Sep 08 13:30:21.308577 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.308558    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-f56957bd6-hjkvs"]
Sep 08 13:30:21.310135 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.310119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.311938 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.311914    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 08 13:30:21.312040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.311937    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 08 13:30:21.312040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.312016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 08 13:30:21.312269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.312248    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 08 13:30:21.312330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.312252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.312330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.312252    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rtdqb\""
Sep 08 13:30:21.312461 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.312258    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.314606 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.314588    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"]
Sep 08 13:30:21.326770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.326744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f56957bd6-hjkvs"]
Sep 08 13:30:21.331883 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.331860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-default-certificate\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.332075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.331899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.332075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.331915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.332075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.331938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-stats-auth\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.332075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.332003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzgs\" (UniqueName: \"kubernetes.io/projected/a571c7c2-e643-490f-9015-04851cb33359-kube-api-access-6qzgs\") pod \"volume-data-source-validator-695c895bd6-5894r\" (UID: \"a571c7c2-e643-490f-9015-04851cb33359\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"
Sep 08 13:30:21.332075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.332054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zptbh\" (UniqueName: \"kubernetes.io/projected/3b045e1d-7b5b-4cf8-b851-56b866652bef-kube-api-access-zptbh\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.407308 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.407268    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"]
Sep 08 13:30:21.409107 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.409078    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"]
Sep 08 13:30:21.409247 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.409212    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.412057 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.412031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 08 13:30:21.412567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.412546    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.412965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.412940    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 08 13:30:21.413094 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.413081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.413228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.413211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wbb9c\""
Sep 08 13:30:21.413772 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.413599    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"]
Sep 08 13:30:21.414211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.413896    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.415980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.415961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.416076 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.416027    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.416245 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.416224    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 08 13:30:21.416717 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.416701    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rdnbg\""
Sep 08 13:30:21.416790 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.416715    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.418991 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.418969    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.419129 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.419001    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 08 13:30:21.419129 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.419045    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 08 13:30:21.419312 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.419243    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8bhqh\""
Sep 08 13:30:21.419312 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.419293    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.420754 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.420735    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"]
Sep 08 13:30:21.421651 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.421629    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"]
Sep 08 13:30:21.423629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.423605    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"]
Sep 08 13:30:21.432537 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-default-certificate\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.432669 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnzvd\" (UniqueName: \"kubernetes.io/projected/c1b39757-ae43-48de-a703-d189056bd6bd-kube-api-access-jnzvd\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.432669 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-stats-auth\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzgs\" (UniqueName: \"kubernetes.io/projected/a571c7c2-e643-490f-9015-04851cb33359-kube-api-access-6qzgs\") pod \"volume-data-source-validator-695c895bd6-5894r\" (UID: \"a571c7c2-e643-490f-9015-04851cb33359\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432861    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zptbh\" (UniqueName: \"kubernetes.io/projected/3b045e1d-7b5b-4cf8-b851-56b866652bef-kube-api-access-zptbh\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.432905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.433244 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.432908    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 13:30:21.433244 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.432928    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njdwc\" (UniqueName: \"kubernetes.io/projected/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-kube-api-access-njdwc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.433244 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.432951    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:21.932931249 +0000 UTC m=+105.249334799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : configmap references non-existent config key: service-ca.crt
Sep 08 13:30:21.433244 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.432982    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:21.932970438 +0000 UTC m=+105.249373983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : secret "router-metrics-certs-default" not found
Sep 08 13:30:21.433543 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.433305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.433543 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.433357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bddqc\" (UniqueName: \"kubernetes.io/projected/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-kube-api-access-bddqc\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.433543 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.433385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.435454 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.435431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-default-certificate\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.435659 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.435641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-stats-auth\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.450164 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.450132    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzgs\" (UniqueName: \"kubernetes.io/projected/a571c7c2-e643-490f-9015-04851cb33359-kube-api-access-6qzgs\") pod \"volume-data-source-validator-695c895bd6-5894r\" (UID: \"a571c7c2-e643-490f-9015-04851cb33359\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"
Sep 08 13:30:21.450308 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.450234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zptbh\" (UniqueName: \"kubernetes.io/projected/3b045e1d-7b5b-4cf8-b851-56b866652bef-kube-api-access-zptbh\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.498463 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.498425    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-q2shb"]
Sep 08 13:30:21.500300 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.500284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.502401 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.502366    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 08 13:30:21.502401 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.502376    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 08 13:30:21.502626 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.502444    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.502713 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.502672    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.502777 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.502693    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-95vn8\""
Sep 08 13:30:21.507787 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.507754    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 08 13:30:21.509373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.509334    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-q2shb"]
Sep 08 13:30:21.533967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.533931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.533984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njdwc\" (UniqueName: \"kubernetes.io/projected/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-kube-api-access-njdwc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bddqc\" (UniqueName: \"kubernetes.io/projected/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-kube-api-access-bddqc\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-config\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.534091    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb9e039-751e-495a-afad-fd5f8ef829c7-serving-cert\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.534143    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:21.534151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp95s\" (UniqueName: \"kubernetes.io/projected/0eb9e039-751e-495a-afad-fd5f8ef829c7-kube-api-access-zp95s\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.534618 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.534175    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls podName:c1b39757-ae43-48de-a703-d189056bd6bd nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.034152571 +0000 UTC m=+105.350556102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-mm86m" (UID: "c1b39757-ae43-48de-a703-d189056bd6bd") : secret "samples-operator-tls" not found
Sep 08 13:30:21.534618 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.534196    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls podName:f24ddbfa-f5f9-48f2-8664-a591f3f2fd18 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.034187431 +0000 UTC m=+105.350590967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-j69xv" (UID: "f24ddbfa-f5f9-48f2-8664-a591f3f2fd18") : secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:21.534618 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-trusted-ca\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.534618 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnzvd\" (UniqueName: \"kubernetes.io/projected/c1b39757-ae43-48de-a703-d189056bd6bd-kube-api-access-jnzvd\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.534618 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.534793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534677    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.534793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.534778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.536868 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.536847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.542844 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.542815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njdwc\" (UniqueName: \"kubernetes.io/projected/61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8-kube-api-access-njdwc\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pmllc\" (UID: \"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.542991 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.542974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnzvd\" (UniqueName: \"kubernetes.io/projected/c1b39757-ae43-48de-a703-d189056bd6bd-kube-api-access-jnzvd\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:21.543066 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.543047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bddqc\" (UniqueName: \"kubernetes.io/projected/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-kube-api-access-bddqc\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:21.613265 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.613174    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"
Sep 08 13:30:21.635121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.635088    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-config\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.635285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.635132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb9e039-751e-495a-afad-fd5f8ef829c7-serving-cert\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.635285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.635158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp95s\" (UniqueName: \"kubernetes.io/projected/0eb9e039-751e-495a-afad-fd5f8ef829c7-kube-api-access-zp95s\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.635285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.635194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-trusted-ca\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.635827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.635806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-config\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.636252 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.636226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0eb9e039-751e-495a-afad-fd5f8ef829c7-trusted-ca\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.637647 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.637629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0eb9e039-751e-495a-afad-fd5f8ef829c7-serving-cert\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.644198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.644169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp95s\" (UniqueName: \"kubernetes.io/projected/0eb9e039-751e-495a-afad-fd5f8ef829c7-kube-api-access-zp95s\") pod \"console-operator-85b454874b-q2shb\" (UID: \"0eb9e039-751e-495a-afad-fd5f8ef829c7\") " pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.735100 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.735060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"
Sep 08 13:30:21.811770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.811738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:21.881533 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.881444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r"]
Sep 08 13:30:21.885434 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:21.885408    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda571c7c2_e643_490f_9015_04851cb33359.slice/crio-fbd293ecaf744ae2f78a044578863ef1e808982b734ae848fc8d7bd32c24953a WatchSource:0}: Error finding container fbd293ecaf744ae2f78a044578863ef1e808982b734ae848fc8d7bd32c24953a: Status 404 returned error can't find the container with id fbd293ecaf744ae2f78a044578863ef1e808982b734ae848fc8d7bd32c24953a
Sep 08 13:30:21.938264 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.938228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.938264 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:21.938264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:21.938528 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.938385    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 13:30:21.938528 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.938431    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.938415435 +0000 UTC m=+106.254818966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : secret "router-metrics-certs-default" not found
Sep 08 13:30:21.938528 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:21.938448    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.938439197 +0000 UTC m=+106.254842729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : configmap references non-existent config key: service-ca.crt
Sep 08 13:30:22.006536 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.006502    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc"]
Sep 08 13:30:22.009725 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:22.009698    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a5ebb7_c1d0_4ad5_b6f4_0980076eeec8.slice/crio-889877c2073169994baf3478cbca75f9d816247dae3165c4399366bc61be330a WatchSource:0}: Error finding container 889877c2073169994baf3478cbca75f9d816247dae3165c4399366bc61be330a: Status 404 returned error can't find the container with id 889877c2073169994baf3478cbca75f9d816247dae3165c4399366bc61be330a
Sep 08 13:30:22.038817 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.038779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:22.038980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.038827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:22.038980 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.038923    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:22.038980 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.038923    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:30:22.038980 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.038972    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls podName:f24ddbfa-f5f9-48f2-8664-a591f3f2fd18 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:23.038960048 +0000 UTC m=+106.355363578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-j69xv" (UID: "f24ddbfa-f5f9-48f2-8664-a591f3f2fd18") : secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:22.039135 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.038985    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls podName:c1b39757-ae43-48de-a703-d189056bd6bd nodeName:}" failed. No retries permitted until 2026-09-08 13:30:23.038979744 +0000 UTC m=+106.355383275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-mm86m" (UID: "c1b39757-ae43-48de-a703-d189056bd6bd") : secret "samples-operator-tls" not found
Sep 08 13:30:22.081364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.081280    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-q2shb"]
Sep 08 13:30:22.085870 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:22.085847    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eb9e039_751e_495a_afad_fd5f8ef829c7.slice/crio-a0b302c2795bce6e81ced2bae00856418ad6ad97eb984506c8f07f5a89d3176b WatchSource:0}: Error finding container a0b302c2795bce6e81ced2bae00856418ad6ad97eb984506c8f07f5a89d3176b: Status 404 returned error can't find the container with id a0b302c2795bce6e81ced2bae00856418ad6ad97eb984506c8f07f5a89d3176b
Sep 08 13:30:22.603727 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.603619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-q2shb" event={"ID":"0eb9e039-751e-495a-afad-fd5f8ef829c7","Type":"ContainerStarted","Data":"a0b302c2795bce6e81ced2bae00856418ad6ad97eb984506c8f07f5a89d3176b"}
Sep 08 13:30:22.605088 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.605053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r" event={"ID":"a571c7c2-e643-490f-9015-04851cb33359","Type":"ContainerStarted","Data":"fbd293ecaf744ae2f78a044578863ef1e808982b734ae848fc8d7bd32c24953a"}
Sep 08 13:30:22.606206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.606181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc" event={"ID":"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8","Type":"ContainerStarted","Data":"889877c2073169994baf3478cbca75f9d816247dae3165c4399366bc61be330a"}
Sep 08 13:30:22.945548 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.945512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:22.945723 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:22.945568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:22.945723 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.945699    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:24.945674716 +0000 UTC m=+108.262078262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : configmap references non-existent config key: service-ca.crt
Sep 08 13:30:22.945866 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.945789    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 13:30:22.945927 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:22.945920    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:24.945898917 +0000 UTC m=+108.262302465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : secret "router-metrics-certs-default" not found
Sep 08 13:30:23.046967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:23.046929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:23.047157 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:23.047007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:23.047157 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:23.047127    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:30:23.047299 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:23.047134    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:23.047299 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:23.047245    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls podName:c1b39757-ae43-48de-a703-d189056bd6bd nodeName:}" failed. No retries permitted until 2026-09-08 13:30:25.047222216 +0000 UTC m=+108.363625763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-mm86m" (UID: "c1b39757-ae43-48de-a703-d189056bd6bd") : secret "samples-operator-tls" not found
Sep 08 13:30:23.047299 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:23.047295    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls podName:f24ddbfa-f5f9-48f2-8664-a591f3f2fd18 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:25.047277083 +0000 UTC m=+108.363680618 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-j69xv" (UID: "f24ddbfa-f5f9-48f2-8664-a591f3f2fd18") : secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:23.533645 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:23.533560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r2pp5"
Sep 08 13:30:23.611693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:23.611657    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r" event={"ID":"a571c7c2-e643-490f-9015-04851cb33359","Type":"ContainerStarted","Data":"e4bc9e425f6cc8320f7dff3aed1e214a2b00dd1b9049d2f6e0c96369db0c8228"}
Sep 08 13:30:24.964422 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:24.964373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:24.964422 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:24.964427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:24.965021 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:24.964551    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 13:30:24.965021 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:24.964582    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:28.964554675 +0000 UTC m=+112.280958223 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : configmap references non-existent config key: service-ca.crt
Sep 08 13:30:24.965021 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:24.964613    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:28.964603048 +0000 UTC m=+112.281006589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : secret "router-metrics-certs-default" not found
Sep 08 13:30:25.065109 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.065063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:25.065109 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.065108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:25.065259 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.065195    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:30:25.065259 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.065223    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:25.065259 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.065253    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls podName:c1b39757-ae43-48de-a703-d189056bd6bd nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.065237566 +0000 UTC m=+112.381641097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-mm86m" (UID: "c1b39757-ae43-48de-a703-d189056bd6bd") : secret "samples-operator-tls" not found
Sep 08 13:30:25.065433 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.065273    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls podName:f24ddbfa-f5f9-48f2-8664-a591f3f2fd18 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.065262179 +0000 UTC m=+112.381665710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-j69xv" (UID: "f24ddbfa-f5f9-48f2-8664-a591f3f2fd18") : secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:25.487550 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.487496    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-5894r" podStartSLOduration=3.206464546 podStartE2EDuration="4.487476974s" totalImagesPullingTime="1.281012428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:23.625916184 +0000 UTC m=+106.942319759" watchObservedRunningTime="2026-09-08 13:30:25.487476974 +0000 UTC m=+108.803880527"
Sep 08 13:30:25.487868 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.487843    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sfvwn"]
Sep 08 13:30:25.508132 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.508095    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sfvwn"]
Sep 08 13:30:25.508270 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.508196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.510419 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.510386    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 13:30:25.510657 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.510642    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 13:30:25.510770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.510684    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 13:30:25.510770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.510744    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 13:30:25.510770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.510758    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j78zr\""
Sep 08 13:30:25.569154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.569121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sht9p\" (UniqueName: \"kubernetes.io/projected/15c878ca-0372-4bc5-b902-625461929dbf-kube-api-access-sht9p\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.569154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.569171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15c878ca-0372-4bc5-b902-625461929dbf-crio-socket\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.569396 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.569320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c878ca-0372-4bc5-b902-625461929dbf-data-volume\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.569396 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.569381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.569478 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.569411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.622643 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.622614    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/0.log"
Sep 08 13:30:25.622803 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.622657    2577 generic.go:356] "Generic (PLEG): container finished" podID="0eb9e039-751e-495a-afad-fd5f8ef829c7" containerID="0df234651d5f0f4334bfb82f2076dd865b0d241306c7985ac65f7cc6116eb3ba" exitCode=255
Sep 08 13:30:25.622803 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.622744    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-q2shb" event={"ID":"0eb9e039-751e-495a-afad-fd5f8ef829c7","Type":"ContainerDied","Data":"0df234651d5f0f4334bfb82f2076dd865b0d241306c7985ac65f7cc6116eb3ba"}
Sep 08 13:30:25.623018 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.622992    2577 scope.go:117] "RemoveContainer" containerID="0df234651d5f0f4334bfb82f2076dd865b0d241306c7985ac65f7cc6116eb3ba"
Sep 08 13:30:25.624447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.624423    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc" event={"ID":"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8","Type":"ContainerStarted","Data":"007ccd4a73d4a7c3ffa47ce9287b4c837c7bca31791110968bc1dca281a311f4"}
Sep 08 13:30:25.651987 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.651942    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc" podStartSLOduration=1.610614085 podStartE2EDuration="4.651925238s" totalImagesPullingTime="3.041311153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:25.651267354 +0000 UTC m=+108.967670917" watchObservedRunningTime="2026-09-08 13:30:25.651925238 +0000 UTC m=+108.968328790"
Sep 08 13:30:25.670064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c878ca-0372-4bc5-b902-625461929dbf-data-volume\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670187 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670187 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670286 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.670187    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:25.670286 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.670187    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:25.670286 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.670245    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:26.17022662 +0000 UTC m=+109.486630215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:25.670286 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:25.670261    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:26.170252958 +0000 UTC m=+109.486656489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:25.670490 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sht9p\" (UniqueName: \"kubernetes.io/projected/15c878ca-0372-4bc5-b902-625461929dbf-kube-api-access-sht9p\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670490 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15c878ca-0372-4bc5-b902-625461929dbf-crio-socket\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670490 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15c878ca-0372-4bc5-b902-625461929dbf-data-volume\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.670756 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.670733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15c878ca-0372-4bc5-b902-625461929dbf-crio-socket\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:25.678500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:25.678472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sht9p\" (UniqueName: \"kubernetes.io/projected/15c878ca-0372-4bc5-b902-625461929dbf-kube-api-access-sht9p\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:26.174411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.174373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:26.174411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.174414    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:26.174822 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:26.174493    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:26.174822 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:26.174530    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:26.174822 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:26.174577    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:27.174564098 +0000 UTC m=+110.490967633 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:26.174822 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:26.174589    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:27.174583097 +0000 UTC m=+110.490986628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:26.371741 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.371666    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"]
Sep 08 13:30:26.373604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.373579    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"
Sep 08 13:30:26.375966 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.375942    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 08 13:30:26.376090 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.375966    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:26.376612 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.376594    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-s26jx\""
Sep 08 13:30:26.386119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.386093    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"]
Sep 08 13:30:26.477102 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.477062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz52q\" (UniqueName: \"kubernetes.io/projected/e1d592c7-9383-4352-b307-fdd687269c5c-kube-api-access-zz52q\") pod \"migrator-fdbb8fdff-gmgf5\" (UID: \"e1d592c7-9383-4352-b307-fdd687269c5c\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"
Sep 08 13:30:26.577987 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.577944    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz52q\" (UniqueName: \"kubernetes.io/projected/e1d592c7-9383-4352-b307-fdd687269c5c-kube-api-access-zz52q\") pod \"migrator-fdbb8fdff-gmgf5\" (UID: \"e1d592c7-9383-4352-b307-fdd687269c5c\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"
Sep 08 13:30:26.585532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.585500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz52q\" (UniqueName: \"kubernetes.io/projected/e1d592c7-9383-4352-b307-fdd687269c5c-kube-api-access-zz52q\") pod \"migrator-fdbb8fdff-gmgf5\" (UID: \"e1d592c7-9383-4352-b307-fdd687269c5c\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"
Sep 08 13:30:26.629492 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.629423    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/1.log"
Sep 08 13:30:26.629854 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.629841    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/0.log"
Sep 08 13:30:26.629906 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.629876    2577 generic.go:356] "Generic (PLEG): container finished" podID="0eb9e039-751e-495a-afad-fd5f8ef829c7" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a" exitCode=255
Sep 08 13:30:26.629949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.629908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-q2shb" event={"ID":"0eb9e039-751e-495a-afad-fd5f8ef829c7","Type":"ContainerDied","Data":"42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"}
Sep 08 13:30:26.629984 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.629973    2577 scope.go:117] "RemoveContainer" containerID="0df234651d5f0f4334bfb82f2076dd865b0d241306c7985ac65f7cc6116eb3ba"
Sep 08 13:30:26.630206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.630190    2577 scope.go:117] "RemoveContainer" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"
Sep 08 13:30:26.630426 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:26.630405    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:30:26.682123 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.682092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"
Sep 08 13:30:26.948484 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:26.948450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5"]
Sep 08 13:30:26.951301 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:26.951271    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d592c7_9383_4352_b307_fdd687269c5c.slice/crio-f53b9cc8aedf1f5404c2154c381ccb3cb8eb4041d2c92be4eaeff23480d53644 WatchSource:0}: Error finding container f53b9cc8aedf1f5404c2154c381ccb3cb8eb4041d2c92be4eaeff23480d53644: Status 404 returned error can't find the container with id f53b9cc8aedf1f5404c2154c381ccb3cb8eb4041d2c92be4eaeff23480d53644
Sep 08 13:30:27.184695 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.184665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:27.185040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.184698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:27.185040 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:27.184786    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:27.185040 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:27.184844    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.184828672 +0000 UTC m=+112.501232203 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:27.185040 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:27.184880    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:27.185040 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:27.184927    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.184915508 +0000 UTC m=+112.501319039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:27.636115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.636079    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/1.log"
Sep 08 13:30:27.636478 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.636456    2577 scope.go:117] "RemoveContainer" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"
Sep 08 13:30:27.636653 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:27.636633    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:30:27.637217 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.637197    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5" event={"ID":"e1d592c7-9383-4352-b307-fdd687269c5c","Type":"ContainerStarted","Data":"f53b9cc8aedf1f5404c2154c381ccb3cb8eb4041d2c92be4eaeff23480d53644"}
Sep 08 13:30:27.641014 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:27.640997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4kt5_1f28bf03-0013-4c8b-a313-6aef862af55f/dns-node-resolver/0.log"
Sep 08 13:30:28.240479 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:28.240444    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dpqb_bb16b466-0c5c-4c93-a0ba-d83777864ac7/node-ca/0.log"
Sep 08 13:30:28.641928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:28.641892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5" event={"ID":"e1d592c7-9383-4352-b307-fdd687269c5c","Type":"ContainerStarted","Data":"5550ba705f52dc27708c83da09eb92af4b45cedcf0149bf774e3fc557851b495"}
Sep 08 13:30:28.642089 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:28.641942    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5" event={"ID":"e1d592c7-9383-4352-b307-fdd687269c5c","Type":"ContainerStarted","Data":"32259b9ae1b10fe3f8d4185f9d1f8098aa99c95876ea3ccf93f7e2b653f2fee2"}
Sep 08 13:30:28.999118 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:28.999010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:28.999118 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:28.999056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:28.999335 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:28.999157    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 08 13:30:28.999335 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:28.999211    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:36.999197125 +0000 UTC m=+120.315600657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : secret "router-metrics-certs-default" not found
Sep 08 13:30:28.999335 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:28.999224    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle podName:3b045e1d-7b5b-4cf8-b851-56b866652bef nodeName:}" failed. No retries permitted until 2026-09-08 13:30:36.999217312 +0000 UTC m=+120.315620843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle") pod "router-default-f56957bd6-hjkvs" (UID: "3b045e1d-7b5b-4cf8-b851-56b866652bef") : configmap references non-existent config key: service-ca.crt
Sep 08 13:30:29.099677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:29.099638    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:29.099822 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:29.099699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:29.099822 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.099795    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 08 13:30:29.099918 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.099852    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:29.099918 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.099861    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls podName:c1b39757-ae43-48de-a703-d189056bd6bd nodeName:}" failed. No retries permitted until 2026-09-08 13:30:37.099840354 +0000 UTC m=+120.416243884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-mm86m" (UID: "c1b39757-ae43-48de-a703-d189056bd6bd") : secret "samples-operator-tls" not found
Sep 08 13:30:29.099984 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.099926    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls podName:f24ddbfa-f5f9-48f2-8664-a591f3f2fd18 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:37.099911785 +0000 UTC m=+120.416315315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-j69xv" (UID: "f24ddbfa-f5f9-48f2-8664-a591f3f2fd18") : secret "cluster-monitoring-operator-tls" not found
Sep 08 13:30:29.200310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:29.200277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:29.200310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:29.200311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:29.200564 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.200420    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:29.200564 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.200436    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:29.200564 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.200482    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:33.200469355 +0000 UTC m=+116.516872885 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:29.200564 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:29.200495    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:33.20048886 +0000 UTC m=+116.516892391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:29.440887 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:29.440858    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5nwm_07486108-6f0e-4822-a7bd-aedfc71b4112/serve-healthcheck-canary/0.log"
Sep 08 13:30:31.812460 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:31.812404    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:31.812860 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:31.812489    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:31.812860 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:31.812805    2577 scope.go:117] "RemoveContainer" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"
Sep 08 13:30:31.812957 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:31.812939    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:30:33.230728 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:33.230688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:33.230728 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:33.230727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:33.231268 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:33.230837    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:33.231268 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:33.230848    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:33.231268 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:33.230897    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:41.23088443 +0000 UTC m=+124.547287961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:33.231268 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:33.230910    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm podName:15c878ca-0372-4bc5-b902-625461929dbf nodeName:}" failed. No retries permitted until 2026-09-08 13:30:41.230903552 +0000 UTC m=+124.547307082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sfvwn" (UID: "15c878ca-0372-4bc5-b902-625461929dbf") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:37.063199 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.063154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:37.063199 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.063203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:37.063847 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.063825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b045e1d-7b5b-4cf8-b851-56b866652bef-service-ca-bundle\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:37.065736 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.065705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b045e1d-7b5b-4cf8-b851-56b866652bef-metrics-certs\") pod \"router-default-f56957bd6-hjkvs\" (UID: \"3b045e1d-7b5b-4cf8-b851-56b866652bef\") " pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:37.163745 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.163701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:37.163925 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.163774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:37.166423 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.166394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c1b39757-ae43-48de-a703-d189056bd6bd-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-mm86m\" (UID: \"c1b39757-ae43-48de-a703-d189056bd6bd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:37.166675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.166652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24ddbfa-f5f9-48f2-8664-a591f3f2fd18-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-j69xv\" (UID: \"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:37.222745 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.222713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rtdqb\""
Sep 08 13:30:37.231782 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.231759    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:37.324761 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.324675    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wbb9c\""
Sep 08 13:30:37.332022 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.331995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rdnbg\""
Sep 08 13:30:37.333003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.332987    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"
Sep 08 13:30:37.341086 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.341070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"
Sep 08 13:30:37.507409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.507322    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-gmgf5" podStartSLOduration=10.138456451 podStartE2EDuration="11.507301401s" totalImagesPullingTime="1.36884495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:28.655953549 +0000 UTC m=+111.972357114" watchObservedRunningTime="2026-09-08 13:30:37.507301401 +0000 UTC m=+120.823704991"
Sep 08 13:30:37.507784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.507768    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f56957bd6-hjkvs"]
Sep 08 13:30:37.511666 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:37.511645    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b045e1d_7b5b_4cf8_b851_56b866652bef.slice/crio-7d9911dd518857c56b292c01aa8267fc96cbf3b30979ef85ba86f303f3797c6a WatchSource:0}: Error finding container 7d9911dd518857c56b292c01aa8267fc96cbf3b30979ef85ba86f303f3797c6a: Status 404 returned error can't find the container with id 7d9911dd518857c56b292c01aa8267fc96cbf3b30979ef85ba86f303f3797c6a
Sep 08 13:30:37.608534 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.608460    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv"]
Sep 08 13:30:37.621241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.621211    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m"]
Sep 08 13:30:37.669704 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.669663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m" event={"ID":"c1b39757-ae43-48de-a703-d189056bd6bd","Type":"ContainerStarted","Data":"ced0875ce200473a4f11074f0f3926bda84aab37eda2553ab22848ff2e4e5fe8"}
Sep 08 13:30:37.670823 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.670796    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv" event={"ID":"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18","Type":"ContainerStarted","Data":"0408a089746c1bfe09d3c1ea340de82a67bff2bf4b25f45f0865b4ac44d94c90"}
Sep 08 13:30:37.672209 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.672188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f56957bd6-hjkvs" event={"ID":"3b045e1d-7b5b-4cf8-b851-56b866652bef","Type":"ContainerStarted","Data":"50f6afc97364b6769df32ce00e754fdd3a4d9188a68ff8de23d8d601d2b41aef"}
Sep 08 13:30:37.672262 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.672226    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f56957bd6-hjkvs" event={"ID":"3b045e1d-7b5b-4cf8-b851-56b866652bef","Type":"ContainerStarted","Data":"7d9911dd518857c56b292c01aa8267fc96cbf3b30979ef85ba86f303f3797c6a"}
Sep 08 13:30:37.688604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:37.688553    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-f56957bd6-hjkvs" podStartSLOduration=16.688535202 podStartE2EDuration="16.688535202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:37.68756724 +0000 UTC m=+121.003970820" watchObservedRunningTime="2026-09-08 13:30:37.688535202 +0000 UTC m=+121.004938755"
Sep 08 13:30:38.232825 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:38.232787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:38.235385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:38.235336    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:38.675106 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:38.675071    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:38.676318 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:38.676296    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-f56957bd6-hjkvs"
Sep 08 13:30:41.299458 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.299415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:41.299879 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.299476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:41.300389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.300362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15c878ca-0372-4bc5-b902-625461929dbf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:41.302165 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.302138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15c878ca-0372-4bc5-b902-625461929dbf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sfvwn\" (UID: \"15c878ca-0372-4bc5-b902-625461929dbf\") " pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:41.419027 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.418999    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j78zr\""
Sep 08 13:30:41.427578 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.427557    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sfvwn"
Sep 08 13:30:41.686448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.686408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m" event={"ID":"c1b39757-ae43-48de-a703-d189056bd6bd","Type":"ContainerStarted","Data":"f303daec37d5b596a89ddcd8a8b51a45a60ec8d79166931981d940233a8a1827"}
Sep 08 13:30:41.686610 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.686474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m" event={"ID":"c1b39757-ae43-48de-a703-d189056bd6bd","Type":"ContainerStarted","Data":"94c18d80cefcd849635bd996ce17af1113200b170c8b5af6de770c68818b58f4"}
Sep 08 13:30:41.702687 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.702642    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-mm86m" podStartSLOduration=16.887682363 podStartE2EDuration="20.702625807s" totalImagesPullingTime="3.814943444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:41.701408881 +0000 UTC m=+125.017812445" watchObservedRunningTime="2026-09-08 13:30:41.702625807 +0000 UTC m=+125.019029360"
Sep 08 13:30:41.715723 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:41.715686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sfvwn"]
Sep 08 13:30:41.718744 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:41.718720    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c878ca_0372_4bc5_b902_625461929dbf.slice/crio-67c5d75a78bfd3e62e82e6f43c5ed523bf0b3d4dcace82d4282aea2aa3bfe71b WatchSource:0}: Error finding container 67c5d75a78bfd3e62e82e6f43c5ed523bf0b3d4dcace82d4282aea2aa3bfe71b: Status 404 returned error can't find the container with id 67c5d75a78bfd3e62e82e6f43c5ed523bf0b3d4dcace82d4282aea2aa3bfe71b
Sep 08 13:30:42.691546 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:42.691508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sfvwn" event={"ID":"15c878ca-0372-4bc5-b902-625461929dbf","Type":"ContainerStarted","Data":"bce564f41aa9bd1821ca766be67d721515f618f9c9d936b64b4a5c8dbdf9938a"}
Sep 08 13:30:42.691926 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:42.691575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sfvwn" event={"ID":"15c878ca-0372-4bc5-b902-625461929dbf","Type":"ContainerStarted","Data":"67c5d75a78bfd3e62e82e6f43c5ed523bf0b3d4dcace82d4282aea2aa3bfe71b"}
Sep 08 13:30:43.207434 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.207400    2577 scope.go:117] "RemoveContainer" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"
Sep 08 13:30:43.696302 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.696276    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:30:43.696773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.696721    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/1.log"
Sep 08 13:30:43.696773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.696757    2577 generic.go:356] "Generic (PLEG): container finished" podID="0eb9e039-751e-495a-afad-fd5f8ef829c7" containerID="cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3" exitCode=255
Sep 08 13:30:43.696841 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.696812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-q2shb" event={"ID":"0eb9e039-751e-495a-afad-fd5f8ef829c7","Type":"ContainerDied","Data":"cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3"}
Sep 08 13:30:43.696871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.696862    2577 scope.go:117] "RemoveContainer" containerID="42bb33ae65d5fc63c982715f25fdf2418dadfef728100e590449b44da9f19e3a"
Sep 08 13:30:43.697190 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:43.697167    2577 scope.go:117] "RemoveContainer" containerID="cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3"
Sep 08 13:30:43.697457 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:43.697439    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:30:44.701692 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:44.701661    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:30:46.943119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:46.943077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:30:46.945649 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:46.945618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/882c9adf-dd13-46b6-9471-dbafc81deb3c-metrics-certs\") pod \"network-metrics-daemon-jvplh\" (UID: \"882c9adf-dd13-46b6-9471-dbafc81deb3c\") " pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:30:47.220886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:47.220805    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56h6n\""
Sep 08 13:30:47.229476 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:47.229453    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jvplh"
Sep 08 13:30:47.505399 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:47.505305    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jvplh"]
Sep 08 13:30:47.508785 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:47.508755    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882c9adf_dd13_46b6_9471_dbafc81deb3c.slice/crio-f530b42e6d90d3e524232353211c75f04c5705562d9f267c7509c02d526b524b WatchSource:0}: Error finding container f530b42e6d90d3e524232353211c75f04c5705562d9f267c7509c02d526b524b: Status 404 returned error can't find the container with id f530b42e6d90d3e524232353211c75f04c5705562d9f267c7509c02d526b524b
Sep 08 13:30:47.714537 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:47.714496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvplh" event={"ID":"882c9adf-dd13-46b6-9471-dbafc81deb3c","Type":"ContainerStarted","Data":"f530b42e6d90d3e524232353211c75f04c5705562d9f267c7509c02d526b524b"}
Sep 08 13:30:47.716236 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:47.716205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv" event={"ID":"f24ddbfa-f5f9-48f2-8664-a591f3f2fd18","Type":"ContainerStarted","Data":"e28eb3c11c0ac3c106e98ea23fd11f2f4166695c649520ffc1ea90ccd2ceecef"}
Sep 08 13:30:48.721299 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:48.721266    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sfvwn" event={"ID":"15c878ca-0372-4bc5-b902-625461929dbf","Type":"ContainerStarted","Data":"55f78781b80dddabed5f505190f619818dc95132e43ed6aff3b65e4768cb2412"}
Sep 08 13:30:49.090985 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.090922    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-j69xv" podStartSLOduration=18.697939712 podStartE2EDuration="28.090896793s" totalImagesPullingTime="9.392957081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:47.732358466 +0000 UTC m=+131.048762034" watchObservedRunningTime="2026-09-08 13:30:49.090896793 +0000 UTC m=+132.407300347"
Sep 08 13:30:49.091992 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.091963    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"]
Sep 08 13:30:49.095083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.095064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:49.098198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.098170    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-89snk\""
Sep 08 13:30:49.098521 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.098505    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 08 13:30:49.104994 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.104970    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"]
Sep 08 13:30:49.159986 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.159943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5c67d\" (UID: \"9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:49.261389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.261328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5c67d\" (UID: \"9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:49.263923 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.263902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-5c67d\" (UID: \"9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:49.403994 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.403962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:49.673868 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.673794    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"]
Sep 08 13:30:49.676898 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:49.676871    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca704a8_1f03_4ed1_aae6_0f6eb9b9420d.slice/crio-e619b2b81f1fe50a6052a2733d888688a0fac27d22b96b0c5933ecc1671b7796 WatchSource:0}: Error finding container e619b2b81f1fe50a6052a2733d888688a0fac27d22b96b0c5933ecc1671b7796: Status 404 returned error can't find the container with id e619b2b81f1fe50a6052a2733d888688a0fac27d22b96b0c5933ecc1671b7796
Sep 08 13:30:49.725540 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:49.725505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d" event={"ID":"9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d","Type":"ContainerStarted","Data":"e619b2b81f1fe50a6052a2733d888688a0fac27d22b96b0c5933ecc1671b7796"}
Sep 08 13:30:50.731071 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:50.730971    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvplh" event={"ID":"882c9adf-dd13-46b6-9471-dbafc81deb3c","Type":"ContainerStarted","Data":"b6d9fd5ca01559baa9b2c7c50c8740b32d062f23eaef5a5aa1a1751b2dbe2b22"}
Sep 08 13:30:50.731071 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:50.731056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jvplh" event={"ID":"882c9adf-dd13-46b6-9471-dbafc81deb3c","Type":"ContainerStarted","Data":"82f2f46979f55d128805681f791b3afdd145d005bd5220f92c51f1c6321fb5c5"}
Sep 08 13:30:50.746090 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:50.746045    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jvplh" podStartSLOduration=130.77442427 podStartE2EDuration="2m13.746029836s" totalImagesPullingTime="2.971605566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:50.745697024 +0000 UTC m=+134.062100588" watchObservedRunningTime="2026-09-08 13:30:50.746029836 +0000 UTC m=+134.062433388"
Sep 08 13:30:51.812826 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:51.812783    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:51.813282 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:51.812862    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:30:51.813282 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:51.813228    2577 scope.go:117] "RemoveContainer" containerID="cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3"
Sep 08 13:30:51.813452 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:30:51.813427    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:30:52.741247 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:52.741209    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sfvwn" event={"ID":"15c878ca-0372-4bc5-b902-625461929dbf","Type":"ContainerStarted","Data":"31a8c078a7f9a6257c54e185c4cd492fac62c598506e070a1ff4729c66c5eb9d"}
Sep 08 13:30:52.757979 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:52.757924    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sfvwn" podStartSLOduration=17.43926935 podStartE2EDuration="27.757908704s" totalImagesPullingTime="10.318639354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:30:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:52.756165081 +0000 UTC m=+136.072568670" watchObservedRunningTime="2026-09-08 13:30:52.757908704 +0000 UTC m=+136.074312257"
Sep 08 13:30:53.746807 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:53.746777    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d" event={"ID":"9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d","Type":"ContainerStarted","Data":"5e81688aca9a84d113806506871ec506bcc27e3688c64181edb6d28dfa3294ec"}
Sep 08 13:30:53.747200 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:53.747082    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:53.748161 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:53.748134    2577 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-78f599db79-5c67d container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.134.0.16:8443/healthz\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body=
Sep 08 13:30:53.748226 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:53.748190    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d" podUID="9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.134.0.16:8443/healthz\": dial tcp 10.134.0.16:8443: connect: connection refused"
Sep 08 13:30:53.760521 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:53.760473    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d" podStartSLOduration=0.753465369 podStartE2EDuration="4.760456482s" totalImagesPullingTime="4.006991113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:53.759588261 +0000 UTC m=+137.075991824" watchObservedRunningTime="2026-09-08 13:30:53.760456482 +0000 UTC m=+137.076860035"
Sep 08 13:30:54.753969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:54.753938    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-5c67d"
Sep 08 13:30:55.550160 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.550126    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"]
Sep 08 13:30:55.553189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.553175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.555144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.555125    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 08 13:30:55.555786 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.555758    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 08 13:30:55.555863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.555801    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 13:30:55.555930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.555904    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bj5n7\""
Sep 08 13:30:55.563274 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.563252    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"]
Sep 08 13:30:55.714585 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.714544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.714585 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.714588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.714823 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.714606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.714823 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.714725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf756\" (UniqueName: \"kubernetes.io/projected/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-kube-api-access-wf756\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.815915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.815815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf756\" (UniqueName: \"kubernetes.io/projected/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-kube-api-access-wf756\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.815915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.815883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.815915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.815905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.815915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.815921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.816699 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.816679    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.818590 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.818566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.818685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.818572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.823228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.823203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf756\" (UniqueName: \"kubernetes.io/projected/9575bc9c-bbc7-4a5f-a406-3275555fd8ca-kube-api-access-wf756\") pod \"prometheus-operator-6f4946db56-gcnxd\" (UID: \"9575bc9c-bbc7-4a5f-a406-3275555fd8ca\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:55.861709 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:55.861666    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"
Sep 08 13:30:56.131738 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:56.131665    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-gcnxd"]
Sep 08 13:30:56.134518 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:30:56.134483    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9575bc9c_bbc7_4a5f_a406_3275555fd8ca.slice/crio-76350f15fc990b9685ebcad3f2335548b2e53c2fe1d9f2a90bdbe0d0d8bcdab1 WatchSource:0}: Error finding container 76350f15fc990b9685ebcad3f2335548b2e53c2fe1d9f2a90bdbe0d0d8bcdab1: Status 404 returned error can't find the container with id 76350f15fc990b9685ebcad3f2335548b2e53c2fe1d9f2a90bdbe0d0d8bcdab1
Sep 08 13:30:56.761678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:56.761633    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd" event={"ID":"9575bc9c-bbc7-4a5f-a406-3275555fd8ca","Type":"ContainerStarted","Data":"76350f15fc990b9685ebcad3f2335548b2e53c2fe1d9f2a90bdbe0d0d8bcdab1"}
Sep 08 13:30:58.770895 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:58.770855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd" event={"ID":"9575bc9c-bbc7-4a5f-a406-3275555fd8ca","Type":"ContainerStarted","Data":"1c00f100cd0aa5cda7f7579a9ef898875c077cc411a9fbfec53fec53b95ed16a"}
Sep 08 13:30:58.771245 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:58.770910    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd" event={"ID":"9575bc9c-bbc7-4a5f-a406-3275555fd8ca","Type":"ContainerStarted","Data":"b660bbea94e9ffd97395da439deed27f05ca9961ee8b8827e5b9ba4966f15fe5"}
Sep 08 13:30:58.787435 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:30:58.787372    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-gcnxd" podStartSLOduration=2.213409026 podStartE2EDuration="3.787338566s" totalImagesPullingTime="1.57392954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:58.785617048 +0000 UTC m=+142.102020618" watchObservedRunningTime="2026-09-08 13:30:58.787338566 +0000 UTC m=+142.103742118"
Sep 08 13:31:00.888643 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.888611    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"]
Sep 08 13:31:00.892100 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.892081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:00.894373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.894333    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 08 13:31:00.894599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.894573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 08 13:31:00.894718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.894573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7qngk\""
Sep 08 13:31:00.903419 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.903391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"]
Sep 08 13:31:00.916589 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.916559    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-prqgm"]
Sep 08 13:31:00.919850 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.919829    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:00.922772 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.922754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 13:31:00.922885 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.922804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 13:31:00.922885 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.922842    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lm4lr\""
Sep 08 13:31:00.922972 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:00.922959    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 13:31:01.058472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-tls\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-root\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sh5k\" (UniqueName: \"kubernetes.io/projected/1a0099cf-450c-4191-8872-58abc5a6e261-kube-api-access-8sh5k\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a0099cf-450c-4191-8872-58abc5a6e261-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058578    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k4t8\" (UniqueName: \"kubernetes.io/projected/79d15551-ba67-4b0f-9fb6-c8148e502eca-kube-api-access-8k4t8\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.058958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-textfile\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058723    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-wtmp\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-metrics-client-ca\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.058958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.058755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-sys\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.159922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-tls\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.159968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-root\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-root\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160085    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160135    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sh5k\" (UniqueName: \"kubernetes.io/projected/1a0099cf-450c-4191-8872-58abc5a6e261-kube-api-access-8sh5k\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.160201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a0099cf-450c-4191-8872-58abc5a6e261-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.160201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4t8\" (UniqueName: \"kubernetes.io/projected/79d15551-ba67-4b0f-9fb6-c8148e502eca-kube-api-access-8k4t8\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160303    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-textfile\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-wtmp\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-metrics-client-ca\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-sys\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-wtmp\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-accelerators-collector-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-textfile\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/79d15551-ba67-4b0f-9fb6-c8148e502eca-sys\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.160998 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:31:01.160855    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 08 13:31:01.160998 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:31:01.160909    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls podName:1a0099cf-450c-4191-8872-58abc5a6e261 nodeName:}" failed. No retries permitted until 2026-09-08 13:31:01.660891398 +0000 UTC m=+144.977294929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls") pod "openshift-state-metrics-657fd6c54-tgvls" (UID: "1a0099cf-450c-4191-8872-58abc5a6e261") : secret "openshift-state-metrics-tls" not found
Sep 08 13:31:01.160998 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.160969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a0099cf-450c-4191-8872-58abc5a6e261-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.161336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.161267    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79d15551-ba67-4b0f-9fb6-c8148e502eca-metrics-client-ca\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.162675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.162646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-tls\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.163064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.163048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79d15551-ba67-4b0f-9fb6-c8148e502eca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.163133 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.163117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.176239 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.176211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4t8\" (UniqueName: \"kubernetes.io/projected/79d15551-ba67-4b0f-9fb6-c8148e502eca-kube-api-access-8k4t8\") pod \"node-exporter-prqgm\" (UID: \"79d15551-ba67-4b0f-9fb6-c8148e502eca\") " pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.176852 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.176829    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sh5k\" (UniqueName: \"kubernetes.io/projected/1a0099cf-450c-4191-8872-58abc5a6e261-kube-api-access-8sh5k\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.229144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.229111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-prqgm"
Sep 08 13:31:01.240483 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:31:01.240444    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d15551_ba67_4b0f_9fb6_c8148e502eca.slice/crio-cadeba6e089e4383ad926c8f5f2aaff58e2e4e0124553581e278afeaba5ec123 WatchSource:0}: Error finding container cadeba6e089e4383ad926c8f5f2aaff58e2e4e0124553581e278afeaba5ec123: Status 404 returned error can't find the container with id cadeba6e089e4383ad926c8f5f2aaff58e2e4e0124553581e278afeaba5ec123
Sep 08 13:31:01.665246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.665216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.667757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.667736    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a0099cf-450c-4191-8872-58abc5a6e261-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-tgvls\" (UID: \"1a0099cf-450c-4191-8872-58abc5a6e261\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:01.782253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.782224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prqgm" event={"ID":"79d15551-ba67-4b0f-9fb6-c8148e502eca","Type":"ContainerStarted","Data":"cadeba6e089e4383ad926c8f5f2aaff58e2e4e0124553581e278afeaba5ec123"}
Sep 08 13:31:01.800542 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:01.800517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"
Sep 08 13:31:02.074871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:02.074798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls"]
Sep 08 13:31:02.077861 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:31:02.077839    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0099cf_450c_4191_8872_58abc5a6e261.slice/crio-ca7635b9196fe11892729941ed66e94213e8d098b3a76206d8b86a3f1e88473a WatchSource:0}: Error finding container ca7635b9196fe11892729941ed66e94213e8d098b3a76206d8b86a3f1e88473a: Status 404 returned error can't find the container with id ca7635b9196fe11892729941ed66e94213e8d098b3a76206d8b86a3f1e88473a
Sep 08 13:31:02.788948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:02.788905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls" event={"ID":"1a0099cf-450c-4191-8872-58abc5a6e261","Type":"ContainerStarted","Data":"bad93564c89503b8533f570ced3b82efb2c996a264fbdbb9c2a5850d22a791a3"}
Sep 08 13:31:02.789121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:02.788969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls" event={"ID":"1a0099cf-450c-4191-8872-58abc5a6e261","Type":"ContainerStarted","Data":"251734f5fcfbff4a4f2f6aa01dd15c64a4c113e2baefc76aedeaae5137d21483"}
Sep 08 13:31:02.789121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:02.788992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls" event={"ID":"1a0099cf-450c-4191-8872-58abc5a6e261","Type":"ContainerStarted","Data":"ca7635b9196fe11892729941ed66e94213e8d098b3a76206d8b86a3f1e88473a"}
Sep 08 13:31:03.207279 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:03.207240    2577 scope.go:117] "RemoveContainer" containerID="cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3"
Sep 08 13:31:03.207774 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:31:03.207440    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-q2shb_openshift-console-operator(0eb9e039-751e-495a-afad-fd5f8ef829c7)\"" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podUID="0eb9e039-751e-495a-afad-fd5f8ef829c7"
Sep 08 13:31:03.794409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:03.794369    2577 generic.go:356] "Generic (PLEG): container finished" podID="79d15551-ba67-4b0f-9fb6-c8148e502eca" containerID="4f294f6d3f5fa017c079c870f8ef9a89c046afd210e9dd25d6ec15285ca39cbf" exitCode=0
Sep 08 13:31:03.794588 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:03.794460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prqgm" event={"ID":"79d15551-ba67-4b0f-9fb6-c8148e502eca","Type":"ContainerDied","Data":"4f294f6d3f5fa017c079c870f8ef9a89c046afd210e9dd25d6ec15285ca39cbf"}
Sep 08 13:31:04.800432 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:04.800394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls" event={"ID":"1a0099cf-450c-4191-8872-58abc5a6e261","Type":"ContainerStarted","Data":"bb93faf32312cf01faf674dea08855862266530d3bec576f4cec8bfbc50ced95"}
Sep 08 13:31:04.802495 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:04.802470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prqgm" event={"ID":"79d15551-ba67-4b0f-9fb6-c8148e502eca","Type":"ContainerStarted","Data":"451447af93726fac825727ea8884136f242a115adac77eee498a5ac4ae8d3f58"}
Sep 08 13:31:04.802621 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:04.802522    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-prqgm" event={"ID":"79d15551-ba67-4b0f-9fb6-c8148e502eca","Type":"ContainerStarted","Data":"c677bb0852d8ee44251facd574704b8ebde4770412c388a62e70cdc24f80dffc"}
Sep 08 13:31:04.816721 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:04.816628    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-tgvls" podStartSLOduration=2.492261708 podStartE2EDuration="4.816612463s" totalImagesPullingTime="2.324350755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:04.815973899 +0000 UTC m=+148.132377474" watchObservedRunningTime="2026-09-08 13:31:04.816612463 +0000 UTC m=+148.133016015"
Sep 08 13:31:04.833186 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:04.833120    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-prqgm" podStartSLOduration=3.044187255 podStartE2EDuration="4.833105202s" totalImagesPullingTime="1.720377899s" totalInitContainerRuntime="68.540048ms" isStatefulPod=true podCreationTimestamp="2026-09-08 13:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:04.832655496 +0000 UTC m=+148.149059096" watchObservedRunningTime="2026-09-08 13:31:04.833105202 +0000 UTC m=+148.149508754"
Sep 08 13:31:05.198860 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.198831    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-684b46cf8f-ps7wt"]
Sep 08 13:31:05.202237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.202215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.204363 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.204327    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 08 13:31:05.204468 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.204327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 08 13:31:05.205020 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.205000    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 08 13:31:05.205122 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.205036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dtrbw\""
Sep 08 13:31:05.205190 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.205156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1skgur28kcji6\""
Sep 08 13:31:05.205251 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.205232    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 08 13:31:05.212914 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.212892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-684b46cf8f-ps7wt"]
Sep 08 13:31:05.395115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8xcm\" (UniqueName: \"kubernetes.io/projected/d50f17c3-94d1-492d-af02-24441997414e-kube-api-access-t8xcm\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d50f17c3-94d1-492d-af02-24441997414e-audit-log\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-metrics-server-audit-profiles\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-client-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-tls\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-client-certs\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.395410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.395329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496316    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-client-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-tls\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-client-certs\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8xcm\" (UniqueName: \"kubernetes.io/projected/d50f17c3-94d1-492d-af02-24441997414e-kube-api-access-t8xcm\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d50f17c3-94d1-492d-af02-24441997414e-audit-log\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.496685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-metrics-server-audit-profiles\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.497001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.496970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d50f17c3-94d1-492d-af02-24441997414e-audit-log\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.497479 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.497456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.497574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.497525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d50f17c3-94d1-492d-af02-24441997414e-metrics-server-audit-profiles\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.499078 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.499052    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-tls\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.499301 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.499281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-client-ca-bundle\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.499369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.499281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d50f17c3-94d1-492d-af02-24441997414e-secret-metrics-server-client-certs\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.504590 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.504570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8xcm\" (UniqueName: \"kubernetes.io/projected/d50f17c3-94d1-492d-af02-24441997414e-kube-api-access-t8xcm\") pod \"metrics-server-684b46cf8f-ps7wt\" (UID: \"d50f17c3-94d1-492d-af02-24441997414e\") " pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.511453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.511433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:05.786940 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.786864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-684b46cf8f-ps7wt"]
Sep 08 13:31:05.791407 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:31:05.791367    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50f17c3_94d1_492d_af02_24441997414e.slice/crio-6586214dd46df86ad1e35e8b74398a35341fbd8ddb5c70cbc313625f3a0938cf WatchSource:0}: Error finding container 6586214dd46df86ad1e35e8b74398a35341fbd8ddb5c70cbc313625f3a0938cf: Status 404 returned error can't find the container with id 6586214dd46df86ad1e35e8b74398a35341fbd8ddb5c70cbc313625f3a0938cf
Sep 08 13:31:05.806669 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:05.806638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt" event={"ID":"d50f17c3-94d1-492d-af02-24441997414e","Type":"ContainerStarted","Data":"6586214dd46df86ad1e35e8b74398a35341fbd8ddb5c70cbc313625f3a0938cf"}
Sep 08 13:31:07.044172 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.044137    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:31:07.049454 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.049426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.054844 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.054815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 13:31:07.055560 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.055533    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 13:31:07.055670 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.055622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 13:31:07.055840 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.055822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 13:31:07.056068 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.056052    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5cne2m2soq5il\""
Sep 08 13:31:07.056428 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.056391    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 13:31:07.056516 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.056429    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 13:31:07.056617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.056599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 13:31:07.056825 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.056804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zm6mv\""
Sep 08 13:31:07.057032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.057016    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 13:31:07.057177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.057156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 13:31:07.057440 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.057422    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 13:31:07.057544 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.057079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 13:31:07.059561 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.059541    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 13:31:07.059969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.059947    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 13:31:07.060052 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.059989    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 13:31:07.063562 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.063531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:31:07.211972 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.211938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.211987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212267    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6zh9\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212410 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.212677 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.212638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313319    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313464    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313852    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6zh9\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.313974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.313964    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.314535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.314148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.314535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.314228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.314825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.314903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.315375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316206 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.315845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316644 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.316617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.316720 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.316678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.317240 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.317210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.319920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.319456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.319920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.319697    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.319920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.319785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.319920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.319879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.320175 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.320134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.320175 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.320163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.321144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.320700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.321144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.320768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.321573 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.321532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.324070 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.324046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6zh9\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9\") pod \"prometheus-k8s-0\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.368933 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.368887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:07.659406 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.659367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:31:07.663267 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:31:07.663234    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a6980e4_9e4a_48d1_9442_585d42358e44.slice/crio-e0e486c2fed1e5b693af13c9f4276917a0494494cad2463ee78a0e7908d32951 WatchSource:0}: Error finding container e0e486c2fed1e5b693af13c9f4276917a0494494cad2463ee78a0e7908d32951: Status 404 returned error can't find the container with id e0e486c2fed1e5b693af13c9f4276917a0494494cad2463ee78a0e7908d32951
Sep 08 13:31:07.816468 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:07.816435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"e0e486c2fed1e5b693af13c9f4276917a0494494cad2463ee78a0e7908d32951"}
Sep 08 13:31:09.826000 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:09.825959    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c" exitCode=0
Sep 08 13:31:09.826364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:09.826048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"}
Sep 08 13:31:12.564519 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:31:12.564427    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-nn66g" podUID="98b7648c-e184-4582-a115-a72957ac4884"
Sep 08 13:31:12.841198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:12.841079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt" event={"ID":"d50f17c3-94d1-492d-af02-24441997414e","Type":"ContainerStarted","Data":"e1d372e34ccddd0b9fe83a69fc58fec4f1b1649746b0b709635f486ddb8bb927"}
Sep 08 13:31:12.841198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:12.841119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:12.856522 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:12.856468    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt" podStartSLOduration=1.391749497 podStartE2EDuration="7.85644758s" totalImagesPullingTime="6.464698083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:12.85626303 +0000 UTC m=+156.172666593" watchObservedRunningTime="2026-09-08 13:31:12.85644758 +0000 UTC m=+156.172851146"
Sep 08 13:31:14.207497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:14.207451    2577 scope.go:117] "RemoveContainer" containerID="cf27a1700e12d3f35b075dd40715d96d675aebddadba8003d46303cd0c5beee3"
Sep 08 13:31:15.856897 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.856858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"}
Sep 08 13:31:15.857402 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.856950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"}
Sep 08 13:31:15.859019 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.858989    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:31:15.859118 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.859042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-q2shb" event={"ID":"0eb9e039-751e-495a-afad-fd5f8ef829c7","Type":"ContainerStarted","Data":"74ad1f9e87e6371209273c71beff1e05764d6231e54d419bb396864f177f0409"}
Sep 08 13:31:15.859923 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.859547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:31:15.875376 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:15.875286    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-q2shb" podStartSLOduration=51.915750557 podStartE2EDuration="54.875268922s" totalImagesPullingTime="2.959518365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:15.874629169 +0000 UTC m=+159.191032724" watchObservedRunningTime="2026-09-08 13:31:15.875268922 +0000 UTC m=+159.191672475"
Sep 08 13:31:16.601948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:16.601918    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-q2shb"
Sep 08 13:31:17.411554 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.411518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:17.414567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.414533    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/98b7648c-e184-4582-a115-a72957ac4884-metrics-tls\") pod \"dns-default-nn66g\" (UID: \"98b7648c-e184-4582-a115-a72957ac4884\") " pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:17.644709 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.644669    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwgzv\""
Sep 08 13:31:17.653288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.653257    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:17.871313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.871227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"}
Sep 08 13:31:17.871313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.871297    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"}
Sep 08 13:31:17.871542 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.871327    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"}
Sep 08 13:31:17.871542 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.871441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerStarted","Data":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"}
Sep 08 13:31:17.898705 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.898605    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.453899695 podStartE2EDuration="10.898584157s" totalImagesPullingTime="8.428538959s" totalInitContainerRuntime="16.145503ms" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:17.897603519 +0000 UTC m=+161.214007121" watchObservedRunningTime="2026-09-08 13:31:17.898584157 +0000 UTC m=+161.214987712"
Sep 08 13:31:17.928143 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:17.928122    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nn66g"]
Sep 08 13:31:17.930555 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:31:17.930528    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98b7648c_e184_4582_a115_a72957ac4884.slice/crio-26c067790ba4f4adc18f8f6b0a39bd3eb237525f1e25e43d2804790cc53c9636 WatchSource:0}: Error finding container 26c067790ba4f4adc18f8f6b0a39bd3eb237525f1e25e43d2804790cc53c9636: Status 404 returned error can't find the container with id 26c067790ba4f4adc18f8f6b0a39bd3eb237525f1e25e43d2804790cc53c9636
Sep 08 13:31:18.877755 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:18.877719    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn66g" event={"ID":"98b7648c-e184-4582-a115-a72957ac4884","Type":"ContainerStarted","Data":"26c067790ba4f4adc18f8f6b0a39bd3eb237525f1e25e43d2804790cc53c9636"}
Sep 08 13:31:19.883577 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:19.883491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn66g" event={"ID":"98b7648c-e184-4582-a115-a72957ac4884","Type":"ContainerStarted","Data":"a0e5e694ec5b839776fee27622cc15952dc2e18891cc50f30e36ad77c35e3643"}
Sep 08 13:31:19.883577 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:19.883543    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nn66g" event={"ID":"98b7648c-e184-4582-a115-a72957ac4884","Type":"ContainerStarted","Data":"12b69e2aad5d32bbe55ad0f82ed61bf30108681dd6f07dbc9cf4e2af6ab6c540"}
Sep 08 13:31:19.884016 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:19.883644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:19.901855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:19.901801    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nn66g" podStartSLOduration=129.3536446 podStartE2EDuration="2m10.901784608s" totalImagesPullingTime="1.548140008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:19.900380964 +0000 UTC m=+163.216784537" watchObservedRunningTime="2026-09-08 13:31:19.901784608 +0000 UTC m=+163.218188160"
Sep 08 13:31:22.369479 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:22.369442    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:31:25.512087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:25.512048    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:25.512477 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:25.512111    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:29.889919 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:29.889849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nn66g"
Sep 08 13:31:40.969954 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:40.969922    2577 generic.go:356] "Generic (PLEG): container finished" podID="61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8" containerID="007ccd4a73d4a7c3ffa47ce9287b4c837c7bca31791110968bc1dca281a311f4" exitCode=0
Sep 08 13:31:40.970469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:40.969991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc" event={"ID":"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8","Type":"ContainerDied","Data":"007ccd4a73d4a7c3ffa47ce9287b4c837c7bca31791110968bc1dca281a311f4"}
Sep 08 13:31:40.970469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:40.970305    2577 scope.go:117] "RemoveContainer" containerID="007ccd4a73d4a7c3ffa47ce9287b4c837c7bca31791110968bc1dca281a311f4"
Sep 08 13:31:41.976735 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:41.976698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pmllc" event={"ID":"61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8","Type":"ContainerStarted","Data":"1bc3216dbed3f6014121db6b08e09c69492ffee1c072e61274d86b65fdb71aaf"}
Sep 08 13:31:45.517775 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:45.517747    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:31:45.521897 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:31:45.521870    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-684b46cf8f-ps7wt"
Sep 08 13:32:07.369338 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:07.369292    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:07.386012 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:07.385986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:08.100465 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:08.100435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:25.450968 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.450926    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:25.451588 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.451356    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="prometheus" containerID="cri-o://9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02" gracePeriod=600
Sep 08 13:32:25.452254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.451810    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-thanos" containerID="cri-o://451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7" gracePeriod=600
Sep 08 13:32:25.452254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.451928    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy" containerID="cri-o://455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656" gracePeriod=600
Sep 08 13:32:25.452254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.452008    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-web" containerID="cri-o://b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2" gracePeriod=600
Sep 08 13:32:25.452254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.452084    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="thanos-sidecar" containerID="cri-o://c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6" gracePeriod=600
Sep 08 13:32:25.452254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.452156    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="config-reloader" containerID="cri-o://727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f" gracePeriod=600
Sep 08 13:32:25.709676 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.709609    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:25.755861 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.755830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756046 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.755872    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756046 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.755897    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756046 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.755951    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756219 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756219 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756170    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756219 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756198    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756318    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756368    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756442    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756491    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6zh9\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756566    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756598    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756626    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.756801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.756655    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file\") pod \"7a6980e4-9e4a-48d1-9442-585d42358e44\" (UID: \"7a6980e4-9e4a-48d1-9442-585d42358e44\") "
Sep 08 13:32:25.757318 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.757283    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.759100 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.757734    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.759100 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.759088    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.759757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.759466    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.759757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.759637    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.760231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.759992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.761370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.760982    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:25.761838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.761705    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:32:25.762572 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.762527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.763662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.763626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out" (OuterVolumeSpecName: "config-out") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:32:25.763890 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.763859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:32:25.764037 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.764013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9" (OuterVolumeSpecName: "kube-api-access-n6zh9") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "kube-api-access-n6zh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:32:25.765923 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.765884    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.767024 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.766998    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.767127 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.767018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.767127 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.767040    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.767127 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.767117    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config" (OuterVolumeSpecName: "config") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.768139 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.768107    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.769263 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.769236    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.777960 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.777928    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config" (OuterVolumeSpecName: "web-config") pod "7a6980e4-9e4a-48d1-9442-585d42358e44" (UID: "7a6980e4-9e4a-48d1-9442-585d42358e44"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:25.857980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.857941    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.857980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.857974    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-config-out\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.857980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.857984    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-grpc-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.857993    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6zh9\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-kube-api-access-n6zh9\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858002    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-db\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858010    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-kube-rbac-proxy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858020    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858031    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858039    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858048    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-web-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858056    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858064    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7a6980e4-9e4a-48d1-9442-585d42358e44-tls-assets\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858073    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-metrics-client-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858080    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858087    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-metrics-client-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858095    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858105    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-config\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858112    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858121    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6980e4-9e4a-48d1-9442-585d42358e44-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:25.858211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:25.858129    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7a6980e4-9e4a-48d1-9442-585d42358e44-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:32:26.161693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161653    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7" exitCode=0
Sep 08 13:32:26.161693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161687    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656" exitCode=0
Sep 08 13:32:26.161693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161697    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2" exitCode=0
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161706    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6" exitCode=0
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161713    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f" exitCode=0
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"}
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161748    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161721    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02" exitCode=0
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161816    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.161924 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"}
Sep 08 13:32:26.162183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"}
Sep 08 13:32:26.162183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.161998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"}
Sep 08 13:32:26.162183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.162040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"}
Sep 08 13:32:26.162183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.162081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"}
Sep 08 13:32:26.162183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.162117    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7a6980e4-9e4a-48d1-9442-585d42358e44","Type":"ContainerDied","Data":"e0e486c2fed1e5b693af13c9f4276917a0494494cad2463ee78a0e7908d32951"}
Sep 08 13:32:26.170624 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.170603    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.178918 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.178897    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.184881 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.184853    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:26.187599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.187571    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.189113 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.189095    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:26.195493 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.195475    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.202839 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.202816    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.211385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.211360    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.214567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214542    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:26.214859 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214846    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="init-config-reloader"
Sep 08 13:32:26.214901 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214861    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="init-config-reloader"
Sep 08 13:32:26.214901 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214868    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-web"
Sep 08 13:32:26.214901 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214874    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-web"
Sep 08 13:32:26.214901 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214885    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="prometheus"
Sep 08 13:32:26.214901 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214891    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="prometheus"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214912    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214917    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214930    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="config-reloader"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214935    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="config-reloader"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214941    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="thanos-sidecar"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214945    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="thanos-sidecar"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214951    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-thanos"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.214956    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-thanos"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215010    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-web"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215018    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215027    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="config-reloader"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215034    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="thanos-sidecar"
Sep 08 13:32:26.215043 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215040    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="kube-rbac-proxy-thanos"
Sep 08 13:32:26.215427 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.215048    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" containerName="prometheus"
Sep 08 13:32:26.218665 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.218645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.221065 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221039    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.221173 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221091    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zm6mv\""
Sep 08 13:32:26.221173 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221104    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 08 13:32:26.221173 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221145    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 08 13:32:26.221325 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221223    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 08 13:32:26.221567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221546    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 08 13:32:26.221660 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221581    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 08 13:32:26.221660 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221653    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 08 13:32:26.221769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221671    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 08 13:32:26.221769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 08 13:32:26.221874 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.221729    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.221874 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221839    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 08 13:32:26.221977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221810    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.221977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.221889    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.222083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222017    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5cne2m2soq5il\""
Sep 08 13:32:26.222083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222077    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 08 13:32:26.222458 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.222319    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.222458 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222367    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.222458 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222390    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.222633 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222551    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 08 13:32:26.222898 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.222758    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.222898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222788    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.222898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222807    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.222898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.222840    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 08 13:32:26.223532 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.223503    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.223626 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.223532    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.223626 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.223553    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.224047 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.224022    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.224138 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224055    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.224138 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224076    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.224427 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.224369    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.224427 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.224427 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224420    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.224726 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:32:26.224701    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.224793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224732    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.224793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.224755    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.225096 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.225068    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.225210 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.225099    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.226148 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.225531    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.226148 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.225560    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.226148 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.226031    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.226148 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.226058    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.226882 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.226855    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 08 13:32:26.227483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.227242    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.227483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.227270    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.228254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.228222    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.228338 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.228256    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.229322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.228569    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.229322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.228596    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.229322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229030    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.229322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229053    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.229562 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229401    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.229562 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229423    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.229814 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229729    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.229814 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.229754    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.230483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.230454    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.230603 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.230484    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.230876 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.230856    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 08 13:32:26.230965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.230868    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.230965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.230888    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.231317 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231297    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.231317 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231316    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.231658 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231642    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.231658 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231658    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.231911 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231889    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.231988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.231914    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.232189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232173    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.232245 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232191    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.232452 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232435    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.232500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232463    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.232707 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232687    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.232753 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.232711    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.233035 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233012    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.233092 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233036    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.233276 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233257    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.233318 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233280    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.233571 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233555    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.233625 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233572    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.233820 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233802    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.233898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.233823    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.234064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234048    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.234109 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234065    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.234336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234320    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.234413 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234335    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.234582 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234566    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.234629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234582    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.234799 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234781    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.234845 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.234801    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.235041 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235027    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.235091 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235040    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.235277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235257    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.235328 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235279    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.235543 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235527    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.235601 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235544    2577 scope.go:117] "RemoveContainer" containerID="451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"
Sep 08 13:32:26.235792 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235776    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7"} err="failed to get container status \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": rpc error: code = NotFound desc = could not find container \"451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7\": container with ID starting with 451e804c2048a18b207c77ae1f910afd4340f237f83cf2f5593783c9ac8a84d7 not found: ID does not exist"
Sep 08 13:32:26.235838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.235794    2577 scope.go:117] "RemoveContainer" containerID="455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"
Sep 08 13:32:26.236018 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236000    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656"} err="failed to get container status \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": rpc error: code = NotFound desc = could not find container \"455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656\": container with ID starting with 455e41d38395020a26eeb1321bcdb1f67658d4f20af94c85011c9a328ea1e656 not found: ID does not exist"
Sep 08 13:32:26.236119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236020    2577 scope.go:117] "RemoveContainer" containerID="b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"
Sep 08 13:32:26.236267 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236253    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2"} err="failed to get container status \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": rpc error: code = NotFound desc = could not find container \"b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2\": container with ID starting with b6902f0b699cf87c5d014cdd34762911132b4fa6c0e395e65c6a0d707826c3b2 not found: ID does not exist"
Sep 08 13:32:26.236314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236268    2577 scope.go:117] "RemoveContainer" containerID="c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"
Sep 08 13:32:26.236518 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236500    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6"} err="failed to get container status \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": rpc error: code = NotFound desc = could not find container \"c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6\": container with ID starting with c284edc87bb5799b0460dfee689111372c6f8e30ca186f41a48d7f4190d980d6 not found: ID does not exist"
Sep 08 13:32:26.236518 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236518    2577 scope.go:117] "RemoveContainer" containerID="727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"
Sep 08 13:32:26.236805 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236788    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f"} err="failed to get container status \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": rpc error: code = NotFound desc = could not find container \"727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f\": container with ID starting with 727dd087686b9d4038af88207b757fc02f4a0eed8a84e4c752bb26999bef470f not found: ID does not exist"
Sep 08 13:32:26.236805 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.236802    2577 scope.go:117] "RemoveContainer" containerID="9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"
Sep 08 13:32:26.237099 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.237030    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02"} err="failed to get container status \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": rpc error: code = NotFound desc = could not find container \"9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02\": container with ID starting with 9b073d0ad5a4411468d72b75ce4e74e6526f5634db8c5724e77eaffbf57d0b02 not found: ID does not exist"
Sep 08 13:32:26.237099 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.237045    2577 scope.go:117] "RemoveContainer" containerID="a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"
Sep 08 13:32:26.237362 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.237326    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c"} err="failed to get container status \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": rpc error: code = NotFound desc = could not find container \"a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c\": container with ID starting with a7977e1c6fbefdc99bc22f95db6f5a74bb0f88913dc796b8200f15c8fc6a327c not found: ID does not exist"
Sep 08 13:32:26.241904 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.241875    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:26.261108 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261258 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261258 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261258 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261258 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb585\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-kube-api-access-rb585\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261528    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261597    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.261712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.261664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.362969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.362923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.362969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.362968    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.362985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb585\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-kube-api-access-rb585\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363485 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363382    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363485 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363485 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363585    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.363872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.363815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.364252 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.364050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.364252 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.364161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.364864 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.364530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.364864 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.364563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.365050 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.364906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.366633 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.366535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-web-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.366906    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.367946 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.367708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.369052 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.369032    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config-out\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.369571 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.369555    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-config\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.369938 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.369915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.369992 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.369950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.376555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.376532    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb585\" (UniqueName: \"kubernetes.io/projected/af3629df-4ff7-49ac-9a10-6c8775c3fd5f-kube-api-access-rb585\") pod \"prometheus-k8s-0\" (UID: \"af3629df-4ff7-49ac-9a10-6c8775c3fd5f\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.532557 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.532500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:32:26.839028 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:26.839005    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 08 13:32:26.841020 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:32:26.840996    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3629df_4ff7_49ac_9a10_6c8775c3fd5f.slice/crio-39ba0d9ba9f574084931c21b189fa96435cf9799967de95d8e6f1a87c2937541 WatchSource:0}: Error finding container 39ba0d9ba9f574084931c21b189fa96435cf9799967de95d8e6f1a87c2937541: Status 404 returned error can't find the container with id 39ba0d9ba9f574084931c21b189fa96435cf9799967de95d8e6f1a87c2937541
Sep 08 13:32:27.171062 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:27.170963    2577 generic.go:356] "Generic (PLEG): container finished" podID="af3629df-4ff7-49ac-9a10-6c8775c3fd5f" containerID="5d954085de272df3982a6c57287d13eca4e6300dc690d0b3db5834b30e52a1b9" exitCode=0
Sep 08 13:32:27.171062 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:27.171022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerDied","Data":"5d954085de272df3982a6c57287d13eca4e6300dc690d0b3db5834b30e52a1b9"}
Sep 08 13:32:27.171569 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:27.171549    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"39ba0d9ba9f574084931c21b189fa96435cf9799967de95d8e6f1a87c2937541"}
Sep 08 13:32:27.212250 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:27.212217    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a6980e4-9e4a-48d1-9442-585d42358e44" path="/var/lib/kubelet/pods/7a6980e4-9e4a-48d1-9442-585d42358e44/volumes"
Sep 08 13:32:28.179863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179828    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"225d633d52d0729bab86f9e9ae72074f438ffb5b22f5bc233a1a312777f95af3"}
Sep 08 13:32:28.180246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"71387bcfe17ca085f2c3798e3837580ff45cc8a2ad58d010e47c597f86f97632"}
Sep 08 13:32:28.180246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179921    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"f3adb59180a222e1ddb0fd005c9192ec9443c6a9ddb77416823d99895d401e00"}
Sep 08 13:32:28.180246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"28ebb9ff5aa61fde5ac076ee94ab12a12de0c33e140025aa55cfe14adee40068"}
Sep 08 13:32:28.180246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179974    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"32429214fe5cb538ff24da627d39ef0056c16d8636c058d9cb39a8da163de66d"}
Sep 08 13:32:28.180246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.179997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af3629df-4ff7-49ac-9a10-6c8775c3fd5f","Type":"ContainerStarted","Data":"921c77f3ccca07e2ec1b7bf60ecb60754c49dcb1fd79a4e821231a9a3fd735e8"}
Sep 08 13:32:28.205551 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:28.205455    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.185335339 podStartE2EDuration="2.205435112s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.099773ms" isStatefulPod=false podCreationTimestamp="2026-09-08 13:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:32:28.203757666 +0000 UTC m=+231.520161238" watchObservedRunningTime="2026-09-08 13:32:28.205435112 +0000 UTC m=+231.521838706"
Sep 08 13:32:31.533467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:32:31.533414    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:33:26.533296 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:26.533254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:33:26.549590 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:26.549562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:33:27.430602 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:27.430571    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 08 13:33:37.107238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:37.107207    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:33:37.112279 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:37.112256    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:33:37.120898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:33:37.120699    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 13:36:21.164803 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.164726    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"]
Sep 08 13:36:21.167706 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.167691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.172743 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.172717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 13:36:21.172892 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.172769    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 13:36:21.173653 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.173641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 13:36:21.176559 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.176540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 13:36:21.182378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.182363    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cdfft\""
Sep 08 13:36:21.212468 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.212440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"]
Sep 08 13:36:21.304511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.304473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhbmk\" (UniqueName: \"kubernetes.io/projected/de1c6466-e64c-448c-86c0-07b45e9e8beb-kube-api-access-bhbmk\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.304718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.304524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.304718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.304620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.304718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.304665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.304831 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.304743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406000 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.405965    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.406008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.406049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.406090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhbmk\" (UniqueName: \"kubernetes.io/projected/de1c6466-e64c-448c-86c0-07b45e9e8beb-kube-api-access-bhbmk\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.406137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.406645 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.406618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.408621 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.408590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de1c6466-e64c-448c-86c0-07b45e9e8beb-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.408789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.408768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.408856 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.408771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/de1c6466-e64c-448c-86c0-07b45e9e8beb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.427654 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.427601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhbmk\" (UniqueName: \"kubernetes.io/projected/de1c6466-e64c-448c-86c0-07b45e9e8beb-kube-api-access-bhbmk\") pod \"opendatahub-operator-controller-manager-54685d959b-hl2bt\" (UID: \"de1c6466-e64c-448c-86c0-07b45e9e8beb\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.477644 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.477617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:21.766725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.766644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"]
Sep 08 13:36:21.770606 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:36:21.770581    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1c6466_e64c_448c_86c0_07b45e9e8beb.slice/crio-625dd0c1e715e77d6c4babacb6151fd500d5f56b8aa74e4989faaf2953568508 WatchSource:0}: Error finding container 625dd0c1e715e77d6c4babacb6151fd500d5f56b8aa74e4989faaf2953568508: Status 404 returned error can't find the container with id 625dd0c1e715e77d6c4babacb6151fd500d5f56b8aa74e4989faaf2953568508
Sep 08 13:36:21.772388 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:21.772369    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:36:22.083586 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:22.083502    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt" event={"ID":"de1c6466-e64c-448c-86c0-07b45e9e8beb","Type":"ContainerStarted","Data":"625dd0c1e715e77d6c4babacb6151fd500d5f56b8aa74e4989faaf2953568508"}
Sep 08 13:36:26.107438 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:26.107398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt" event={"ID":"de1c6466-e64c-448c-86c0-07b45e9e8beb","Type":"ContainerStarted","Data":"c642ad56bacb1612e65560cbcc0a04f0290e9ff591bc4ee0ef83fccabbd6fdc4"}
Sep 08 13:36:27.113304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:27.113271    2577 generic.go:356] "Generic (PLEG): container finished" podID="de1c6466-e64c-448c-86c0-07b45e9e8beb" containerID="c642ad56bacb1612e65560cbcc0a04f0290e9ff591bc4ee0ef83fccabbd6fdc4" exitCode=0
Sep 08 13:36:27.113773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:27.113379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt" event={"ID":"de1c6466-e64c-448c-86c0-07b45e9e8beb","Type":"ContainerDied","Data":"c642ad56bacb1612e65560cbcc0a04f0290e9ff591bc4ee0ef83fccabbd6fdc4"}
Sep 08 13:36:29.124299 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:29.124267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt" event={"ID":"de1c6466-e64c-448c-86c0-07b45e9e8beb","Type":"ContainerStarted","Data":"a28cdc2a6ba18419391a80e1648ec779cff6732d6f21b2f90b40e0635f6fee08"}
Sep 08 13:36:29.124702 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:29.124375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:29.171782 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:29.171728    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt" podStartSLOduration=2.12168152 podStartE2EDuration="8.171711659s" totalImagesPullingTime="5.026739639s" totalInitContainerRuntime="1.0232905s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:29.170495331 +0000 UTC m=+472.486898897" watchObservedRunningTime="2026-09-08 13:36:29.171711659 +0000 UTC m=+472.488115212"
Sep 08 13:36:40.131473 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:40.131439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-hl2bt"
Sep 08 13:36:55.548009 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.547974    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"]
Sep 08 13:36:55.562780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.562751    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.565469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.565449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"]
Sep 08 13:36:55.567026 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.567006    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 13:36:55.567026 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.567023    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qf5j9\""
Sep 08 13:36:55.658725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54szm\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-kube-api-access-54szm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658857    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.658958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.658952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760241 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760238    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54szm\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-kube-api-access-54szm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760525 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760525 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760525 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760525 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760740 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760740 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760740 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760653    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760843 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760843 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.760991 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.760970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.761038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.761006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.761253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.761238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.762723 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.762701    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.763028 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.763013    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.772615 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.772596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.772696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.772665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54szm\" (UniqueName: \"kubernetes.io/projected/879b6174-5433-4b2b-bc79-fb4ae24e6d36-kube-api-access-54szm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl\" (UID: \"879b6174-5433-4b2b-bc79-fb4ae24e6d36\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:55.874287 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:55.874204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:56.158097 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:56.158071    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"]
Sep 08 13:36:56.159694 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:36:56.159666    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879b6174_5433_4b2b_bc79_fb4ae24e6d36.slice/crio-00f6778467acd62c54706ae8d9d2a85dcc49948f04b48a007d1959019e72d257 WatchSource:0}: Error finding container 00f6778467acd62c54706ae8d9d2a85dcc49948f04b48a007d1959019e72d257: Status 404 returned error can't find the container with id 00f6778467acd62c54706ae8d9d2a85dcc49948f04b48a007d1959019e72d257
Sep 08 13:36:56.234936 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:56.234901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl" event={"ID":"879b6174-5433-4b2b-bc79-fb4ae24e6d36","Type":"ContainerStarted","Data":"00f6778467acd62c54706ae8d9d2a85dcc49948f04b48a007d1959019e72d257"}
Sep 08 13:36:58.880151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:58.880110    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:36:58.880394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:58.880192    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:36:58.880394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:58.880218    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:36:59.252490 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.252446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl" event={"ID":"879b6174-5433-4b2b-bc79-fb4ae24e6d36","Type":"ContainerStarted","Data":"efa518929f0252428f06ed7408aa364720cd002528172a26ce218d94cb448ed4"}
Sep 08 13:36:59.294801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.294753    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl" podStartSLOduration=1.576572587 podStartE2EDuration="4.294737685s" totalImagesPullingTime="2.718165098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:59.293217307 +0000 UTC m=+502.609620861" watchObservedRunningTime="2026-09-08 13:36:59.294737685 +0000 UTC m=+502.611141238"
Sep 08 13:36:59.328234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.328203    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"]
Sep 08 13:36:59.331430 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.331416    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.336367 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336324    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 08 13:36:59.336509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336367    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:36:59.336509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336399    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rxx8j\""
Sep 08 13:36:59.336638 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336590    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 08 13:36:59.336638 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336623    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 08 13:36:59.336742 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.336709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 13:36:59.348843 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.348822    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"]
Sep 08 13:36:59.393877 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.393850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj8m5\" (UniqueName: \"kubernetes.io/projected/9268d61d-0852-4e4e-bc67-6a6126763720-kube-api-access-dj8m5\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.394003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.393892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9268d61d-0852-4e4e-bc67-6a6126763720-manager-config\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.394003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.393923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.394003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.393974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-metrics-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.494694 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.494659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj8m5\" (UniqueName: \"kubernetes.io/projected/9268d61d-0852-4e4e-bc67-6a6126763720-kube-api-access-dj8m5\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.494902 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.494733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9268d61d-0852-4e4e-bc67-6a6126763720-manager-config\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.494902 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.494782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.494902 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.494821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-metrics-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.495708 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.495626    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9268d61d-0852-4e4e-bc67-6a6126763720-manager-config\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.497374 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.497357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.497488 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.497473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9268d61d-0852-4e4e-bc67-6a6126763720-metrics-cert\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.508739 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.508690    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj8m5\" (UniqueName: \"kubernetes.io/projected/9268d61d-0852-4e4e-bc67-6a6126763720-kube-api-access-dj8m5\") pod \"lws-controller-manager-8d99975cf-kz9cb\" (UID: \"9268d61d-0852-4e4e-bc67-6a6126763720\") " pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.640724 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.640696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:36:59.875399 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.875289    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:59.880385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.880335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:36:59.929979 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:36:59.929955    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"]
Sep 08 13:36:59.932219 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:36:59.932196    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9268d61d_0852_4e4e_bc67_6a6126763720.slice/crio-e36e4241d484c1cc7796c6185757a5ed8fe51e724621e580c0a25020c428aa17 WatchSource:0}: Error finding container e36e4241d484c1cc7796c6185757a5ed8fe51e724621e580c0a25020c428aa17: Status 404 returned error can't find the container with id e36e4241d484c1cc7796c6185757a5ed8fe51e724621e580c0a25020c428aa17
Sep 08 13:37:00.258381 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:00.258328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb" event={"ID":"9268d61d-0852-4e4e-bc67-6a6126763720","Type":"ContainerStarted","Data":"e36e4241d484c1cc7796c6185757a5ed8fe51e724621e580c0a25020c428aa17"}
Sep 08 13:37:00.258608 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:00.258591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:37:00.259647 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:00.259625    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl"
Sep 08 13:37:02.012628 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.012594    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"]
Sep 08 13:37:02.021489 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.021459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.024317 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.024294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 08 13:37:02.024686 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.024656    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 08 13:37:02.024807 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.024666    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 08 13:37:02.024807 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.024683    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hm24d\""
Sep 08 13:37:02.107823 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.107752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"]
Sep 08 13:37:02.119459 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19097394-d232-4cb8-a149-500bff74505b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119582    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19097394-d232-4cb8-a149-500bff74505b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.119827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.119771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv4fk\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-kube-api-access-xv4fk\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221142    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19097394-d232-4cb8-a149-500bff74505b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv4fk\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-kube-api-access-xv4fk\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19097394-d232-4cb8-a149-500bff74505b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.221541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.221405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.222480 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.222455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19097394-d232-4cb8-a149-500bff74505b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.224278 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.224257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.224650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.224622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19097394-d232-4cb8-a149-500bff74505b-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.224754 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.224716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.224894 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.224872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19097394-d232-4cb8-a149-500bff74505b-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.233274 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.233232    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.233893 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.233851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv4fk\" (UniqueName: \"kubernetes.io/projected/19097394-d232-4cb8-a149-500bff74505b-kube-api-access-xv4fk\") pod \"istiod-openshift-gateway-5677ccb49b-9cxwg\" (UID: \"19097394-d232-4cb8-a149-500bff74505b\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:02.333869 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:02.333792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:03.149909 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:03.149881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"]
Sep 08 13:37:03.152937 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:37:03.152901    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19097394_d232_4cb8_a149_500bff74505b.slice/crio-8ffcd7d9a4f13d1b2972e001e21c1dff247cc0bf28423bfce39bfeb6c3cee50e WatchSource:0}: Error finding container 8ffcd7d9a4f13d1b2972e001e21c1dff247cc0bf28423bfce39bfeb6c3cee50e: Status 404 returned error can't find the container with id 8ffcd7d9a4f13d1b2972e001e21c1dff247cc0bf28423bfce39bfeb6c3cee50e
Sep 08 13:37:03.275533 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:03.275495    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg" event={"ID":"19097394-d232-4cb8-a149-500bff74505b","Type":"ContainerStarted","Data":"8ffcd7d9a4f13d1b2972e001e21c1dff247cc0bf28423bfce39bfeb6c3cee50e"}
Sep 08 13:37:03.277067 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:03.277044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb" event={"ID":"9268d61d-0852-4e4e-bc67-6a6126763720","Type":"ContainerStarted","Data":"77c2add7973e0130e68ce17d7bdf7b03b01ce55fbcc262447cf9d60a613c78f0"}
Sep 08 13:37:03.277199 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:03.277169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:37:03.361667 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:03.361618    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb" podStartSLOduration=1.405393952 podStartE2EDuration="4.361602017s" totalImagesPullingTime="2.956208065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:37:03.359436185 +0000 UTC m=+506.675839756" watchObservedRunningTime="2026-09-08 13:37:03.361602017 +0000 UTC m=+506.678005569"
Sep 08 13:37:05.298983 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:05.298947    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:37:05.299256 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:05.299010    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:37:06.293626 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:06.293592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg" event={"ID":"19097394-d232-4cb8-a149-500bff74505b","Type":"ContainerStarted","Data":"35cff7a99991f909ad360e2772642522ed11b12cbf3552b75442cebd77235245"}
Sep 08 13:37:06.293822 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:06.293802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:06.295413 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:06.295331    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-9cxwg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 13:37:06.295534 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:06.295443    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg" podUID="19097394-d232-4cb8-a149-500bff74505b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:37:06.342081 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:06.341902    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg" podStartSLOduration=3.19819178 podStartE2EDuration="5.341879311s" totalImagesPullingTime="2.143687531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:37:06.341029536 +0000 UTC m=+509.657433098" watchObservedRunningTime="2026-09-08 13:37:06.341879311 +0000 UTC m=+509.658282865"
Sep 08 13:37:07.298275 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:07.298248    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9cxwg"
Sep 08 13:37:14.284387 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:37:14.284353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8d99975cf-kz9cb"
Sep 08 13:38:37.137723 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:38:37.137688    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:38:37.145544 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:38:37.145520    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:39:02.985481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:02.985397    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"]
Sep 08 13:39:02.987898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:02.987878    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:02.990390 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:02.990373    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-jkd99\""
Sep 08 13:39:03.002626 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.002600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"]
Sep 08 13:39:03.062268 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062277    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gclxq\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-kube-api-access-gclxq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062304    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0460699c-2133-46e6-b289-148b07e9ce69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0460699c-2133-46e6-b289-148b07e9ce69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.062678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.062593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163714 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0460699c-2133-46e6-b289-148b07e9ce69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163714 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gclxq\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-kube-api-access-gclxq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.163944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0460699c-2133-46e6-b289-148b07e9ce69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164225 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.163963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164225 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.164158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.164318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164417 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.164312    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164506 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.164474    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0460699c-2133-46e6-b289-148b07e9ce69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.164693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.164674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.166256 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.166226    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0460699c-2133-46e6-b289-148b07e9ce69-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.166560 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.166541    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0460699c-2133-46e6-b289-148b07e9ce69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.172102 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.172081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.173827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.173809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gclxq\" (UniqueName: \"kubernetes.io/projected/0460699c-2133-46e6-b289-148b07e9ce69-kube-api-access-gclxq\") pod \"maas-default-gateway-openshift-default-78bb78ff78-vtl2w\" (UID: \"0460699c-2133-46e6-b289-148b07e9ce69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.298370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.298262    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:03.578896 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.578874    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"]
Sep 08 13:39:03.580941 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:39:03.580913    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0460699c_2133_46e6_b289_148b07e9ce69.slice/crio-455dc43cf2294a32f4681b78fcd2d34d9d27b4d7277c0616854b44e1b4e193f1 WatchSource:0}: Error finding container 455dc43cf2294a32f4681b78fcd2d34d9d27b4d7277c0616854b44e1b4e193f1: Status 404 returned error can't find the container with id 455dc43cf2294a32f4681b78fcd2d34d9d27b4d7277c0616854b44e1b4e193f1
Sep 08 13:39:03.583017 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.582977    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:39:03.583079 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.583063    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:39:03.583114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.583104    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:39:03.804398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.804368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" event={"ID":"0460699c-2133-46e6-b289-148b07e9ce69","Type":"ContainerStarted","Data":"b42ec3b35ba2c0e38017c0ab4759c9436d36b2bfd8e71683b063dfd6ffa2c81c"}
Sep 08 13:39:03.804588 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.804417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" event={"ID":"0460699c-2133-46e6-b289-148b07e9ce69","Type":"ContainerStarted","Data":"455dc43cf2294a32f4681b78fcd2d34d9d27b4d7277c0616854b44e1b4e193f1"}
Sep 08 13:39:03.836926 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:03.836821    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podStartSLOduration=1.8368039710000001 podStartE2EDuration="1.836803971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:39:03.835318034 +0000 UTC m=+627.151721612" watchObservedRunningTime="2026-09-08 13:39:03.836803971 +0000 UTC m=+627.153207524"
Sep 08 13:39:04.298563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:04.298532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:04.303655 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:04.303633    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:04.809105 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:04.809067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:04.809910 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:04.809890    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:39:36.985470 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.985436    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"]
Sep 08 13:39:36.987606 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.987591    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:36.990822 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.990796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 13:39:36.991003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.990845    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 13:39:36.991593 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.991576    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 13:39:36.998942 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:36.998925    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-w6qjh\""
Sep 08 13:39:37.015936 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.015913    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"]
Sep 08 13:39:37.058698 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.058665    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b0740d8-33c7-4d83-bb6c-fb21a656448a-data\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.058865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.058722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxfk\" (UniqueName: \"kubernetes.io/projected/5b0740d8-33c7-4d83-bb6c-fb21a656448a-kube-api-access-snxfk\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.159683 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.159651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxfk\" (UniqueName: \"kubernetes.io/projected/5b0740d8-33c7-4d83-bb6c-fb21a656448a-kube-api-access-snxfk\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.159857 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.159738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b0740d8-33c7-4d83-bb6c-fb21a656448a-data\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.160063 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.160042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5b0740d8-33c7-4d83-bb6c-fb21a656448a-data\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.175234 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.175208    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 13:39:37.184672 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.184658    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 13:39:37.194948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.194928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxfk\" (UniqueName: \"kubernetes.io/projected/5b0740d8-33c7-4d83-bb6c-fb21a656448a-kube-api-access-snxfk\") pod \"postgres-69fb4594cf-jjsgk\" (UID: \"5b0740d8-33c7-4d83-bb6c-fb21a656448a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.299732 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.299658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-w6qjh\""
Sep 08 13:39:37.307505 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.307487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:37.597786 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.597760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"]
Sep 08 13:39:37.599671 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:39:37.599640    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b0740d8_33c7_4d83_bb6c_fb21a656448a.slice/crio-5decb3b954639dbbf110ed5ff5857c6430cdb6a94d743f6aeab4f3759b8d4a99 WatchSource:0}: Error finding container 5decb3b954639dbbf110ed5ff5857c6430cdb6a94d743f6aeab4f3759b8d4a99: Status 404 returned error can't find the container with id 5decb3b954639dbbf110ed5ff5857c6430cdb6a94d743f6aeab4f3759b8d4a99
Sep 08 13:39:37.958334 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:37.958300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk" event={"ID":"5b0740d8-33c7-4d83-bb6c-fb21a656448a","Type":"ContainerStarted","Data":"5decb3b954639dbbf110ed5ff5857c6430cdb6a94d743f6aeab4f3759b8d4a99"}
Sep 08 13:39:42.205750 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:42.205727    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 08 13:39:42.986269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:42.986227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk" event={"ID":"5b0740d8-33c7-4d83-bb6c-fb21a656448a","Type":"ContainerStarted","Data":"e7797991247c3e82e3613edf86ba5d6bcd4a575265641b116379f2c15a907b12"}
Sep 08 13:39:42.986470 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:42.986375    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:43.014974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:43.014601    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk" podStartSLOduration=2.413375957 podStartE2EDuration="7.014583061s" totalImagesPullingTime="4.601207104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:39:43.014485845 +0000 UTC m=+666.330889420" watchObservedRunningTime="2026-09-08 13:39:43.014583061 +0000 UTC m=+666.330986615"
Sep 08 13:39:49.019883 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:49.019853    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-jjsgk"
Sep 08 13:39:56.795662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.795631    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:39:56.819856 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.819829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:39:56.820001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.819933    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:56.822385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.822328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 13:39:56.822385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.822328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 13:39:56.822630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.822332    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 13:39:56.822630 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.822386    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t9j22\""
Sep 08 13:39:56.943145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.943114    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:56.943503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.943153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-597lj\" (UniqueName: \"kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:56.943503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.943204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:56.988464 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.988438    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 13:39:56.991964 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:56.991949    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.001780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.001751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 13:39:57.043795 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.043769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-597lj\" (UniqueName: \"kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.043952 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.043823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.043952 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.043884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.046493 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.046435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.046493 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.046465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.054535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.054512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-597lj\" (UniqueName: \"kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj\") pod \"maas-controller-7dd98f69d7-rtgpd\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") " pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.133989 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.133957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:39:57.144536 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.144508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.144671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.144557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbg5\" (UniqueName: \"kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.144671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.144628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.245520 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.245484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbg5\" (UniqueName: \"kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.245706 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.245552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.245706 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.245602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.248139 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.248116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.248248 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.248162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.253500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.253479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbg5\" (UniqueName: \"kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5\") pod \"maas-controller-56cd6bfd4b-6wpmr\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") " pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.303394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.303288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:39:57.406069 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.406043    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:39:57.408124 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:39:57.408094    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108492ba_81e6_4eac_9672_60d49b6652f7.slice/crio-47f3d45a942f3481bdc33968ee55030e6491315076f9efa497ebaeb96641ec22 WatchSource:0}: Error finding container 47f3d45a942f3481bdc33968ee55030e6491315076f9efa497ebaeb96641ec22: Status 404 returned error can't find the container with id 47f3d45a942f3481bdc33968ee55030e6491315076f9efa497ebaeb96641ec22
Sep 08 13:39:57.579827 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:57.579800    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 13:39:57.581734 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:39:57.581706    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13a3bbe_5932_45ac_9871_9328af4a7e7c.slice/crio-3c502aeefd5e472a673f041f15e6a60b6137d476a9db897b748d612375d49874 WatchSource:0}: Error finding container 3c502aeefd5e472a673f041f15e6a60b6137d476a9db897b748d612375d49874: Status 404 returned error can't find the container with id 3c502aeefd5e472a673f041f15e6a60b6137d476a9db897b748d612375d49874
Sep 08 13:39:58.052431 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:58.052390    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" event={"ID":"d13a3bbe-5932-45ac-9871-9328af4a7e7c","Type":"ContainerStarted","Data":"3c502aeefd5e472a673f041f15e6a60b6137d476a9db897b748d612375d49874"}
Sep 08 13:39:58.053547 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:39:58.053527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" event={"ID":"108492ba-81e6-4eac-9672-60d49b6652f7","Type":"ContainerStarted","Data":"47f3d45a942f3481bdc33968ee55030e6491315076f9efa497ebaeb96641ec22"}
Sep 08 13:40:01.072239 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.072126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" event={"ID":"d13a3bbe-5932-45ac-9871-9328af4a7e7c","Type":"ContainerStarted","Data":"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"}
Sep 08 13:40:01.072664 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.072297    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:40:01.073576 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.073556    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" event={"ID":"108492ba-81e6-4eac-9672-60d49b6652f7","Type":"ContainerStarted","Data":"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"}
Sep 08 13:40:01.073687 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.073677    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:40:01.092900 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.092460    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" podStartSLOduration=2.555732122 podStartE2EDuration="5.092442723s" totalImagesPullingTime="2.536710601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:40:01.091405377 +0000 UTC m=+684.407808950" watchObservedRunningTime="2026-09-08 13:40:01.092442723 +0000 UTC m=+684.408846277"
Sep 08 13:40:01.117326 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:01.117277    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" podStartSLOduration=1.873809155 podStartE2EDuration="5.117261922s" totalImagesPullingTime="3.243452767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:40:01.116225999 +0000 UTC m=+684.432629551" watchObservedRunningTime="2026-09-08 13:40:01.117261922 +0000 UTC m=+684.433665511"
Sep 08 13:40:12.080666 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.080626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:40:12.081163 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.080746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 13:40:12.142863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.142832    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:40:12.143075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.143047    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" podUID="108492ba-81e6-4eac-9672-60d49b6652f7" containerName="manager" containerID="cri-o://058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f" gracePeriod=30
Sep 08 13:40:12.392218 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.392185    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:40:12.489580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.489554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-597lj\" (UniqueName: \"kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj\") pod \"108492ba-81e6-4eac-9672-60d49b6652f7\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") "
Sep 08 13:40:12.489768 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.489635    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert\") pod \"108492ba-81e6-4eac-9672-60d49b6652f7\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") "
Sep 08 13:40:12.489768 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.489677    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs\") pod \"108492ba-81e6-4eac-9672-60d49b6652f7\" (UID: \"108492ba-81e6-4eac-9672-60d49b6652f7\") "
Sep 08 13:40:12.491801 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.491774    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "108492ba-81e6-4eac-9672-60d49b6652f7" (UID: "108492ba-81e6-4eac-9672-60d49b6652f7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:40:12.491905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.491848    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj" (OuterVolumeSpecName: "kube-api-access-597lj") pod "108492ba-81e6-4eac-9672-60d49b6652f7" (UID: "108492ba-81e6-4eac-9672-60d49b6652f7"). InnerVolumeSpecName "kube-api-access-597lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:40:12.491947 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.491920    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "108492ba-81e6-4eac-9672-60d49b6652f7" (UID: "108492ba-81e6-4eac-9672-60d49b6652f7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:40:12.590934 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.590904    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-metrics-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:40:12.590934 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.590930    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-597lj\" (UniqueName: \"kubernetes.io/projected/108492ba-81e6-4eac-9672-60d49b6652f7-kube-api-access-597lj\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:40:12.590934 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:12.590940    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/108492ba-81e6-4eac-9672-60d49b6652f7-webhook-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:40:13.135133 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.135095    2577 generic.go:356] "Generic (PLEG): container finished" podID="108492ba-81e6-4eac-9672-60d49b6652f7" containerID="058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f" exitCode=0
Sep 08 13:40:13.135580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.135154    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd"
Sep 08 13:40:13.135580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.135170    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" event={"ID":"108492ba-81e6-4eac-9672-60d49b6652f7","Type":"ContainerDied","Data":"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"}
Sep 08 13:40:13.135580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.135245    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd98f69d7-rtgpd" event={"ID":"108492ba-81e6-4eac-9672-60d49b6652f7","Type":"ContainerDied","Data":"47f3d45a942f3481bdc33968ee55030e6491315076f9efa497ebaeb96641ec22"}
Sep 08 13:40:13.135580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.135277    2577 scope.go:117] "RemoveContainer" containerID="058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"
Sep 08 13:40:13.145793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.145610    2577 scope.go:117] "RemoveContainer" containerID="058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"
Sep 08 13:40:13.145993 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:40:13.145974    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f\": container with ID starting with 058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f not found: ID does not exist" containerID="058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"
Sep 08 13:40:13.146035 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.146011    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f"} err="failed to get container status \"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f\": rpc error: code = NotFound desc = could not find container \"058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f\": container with ID starting with 058103720d764065140a660cb909b2d29ad4c350c7aa9ca6598f61194586350f not found: ID does not exist"
Sep 08 13:40:13.157545 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.157519    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:40:13.160911 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.160890    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dd98f69d7-rtgpd"]
Sep 08 13:40:13.212942 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:13.212912    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="108492ba-81e6-4eac-9672-60d49b6652f7" path="/var/lib/kubelet/pods/108492ba-81e6-4eac-9672-60d49b6652f7/volumes"
Sep 08 13:40:34.477181 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.477137    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"]
Sep 08 13:40:34.477719 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.477672    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="108492ba-81e6-4eac-9672-60d49b6652f7" containerName="manager"
Sep 08 13:40:34.477719 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.477691    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="108492ba-81e6-4eac-9672-60d49b6652f7" containerName="manager"
Sep 08 13:40:34.477861 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.477794    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="108492ba-81e6-4eac-9672-60d49b6652f7" containerName="manager"
Sep 08 13:40:34.481859 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.481838    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.487036 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.487011    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xq7pl\""
Sep 08 13:40:34.487305 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.487283    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 08 13:40:34.505293 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.505270    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"]
Sep 08 13:40:34.555094 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.555048    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"]
Sep 08 13:40:34.557221 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.557202    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.585164 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.585128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjpx\" (UniqueName: \"kubernetes.io/projected/6314af19-a498-4ab0-8110-b5156bc17d3f-kube-api-access-thjpx\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.585387 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.585187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6314af19-a498-4ab0-8110-b5156bc17d3f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.590242 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.590217    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"]
Sep 08 13:40:34.686404 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.686338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.686555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.686427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spvf9\" (UniqueName: \"kubernetes.io/projected/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-kube-api-access-spvf9\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.686555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.686469    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjpx\" (UniqueName: \"kubernetes.io/projected/6314af19-a498-4ab0-8110-b5156bc17d3f-kube-api-access-thjpx\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.686555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.686504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6314af19-a498-4ab0-8110-b5156bc17d3f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.687151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.687133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6314af19-a498-4ab0-8110-b5156bc17d3f-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.700864 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.700832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjpx\" (UniqueName: \"kubernetes.io/projected/6314af19-a498-4ab0-8110-b5156bc17d3f-kube-api-access-thjpx\") pod \"payload-pre-processing-59fbc47bf8-cx6dt\" (UID: \"6314af19-a498-4ab0-8110-b5156bc17d3f\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.788237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.788140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.788237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.788195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spvf9\" (UniqueName: \"kubernetes.io/projected/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-kube-api-access-spvf9\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.788838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.788817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.791895 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.791868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:34.797928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.797895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spvf9\" (UniqueName: \"kubernetes.io/projected/8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8-kube-api-access-spvf9\") pod \"payload-processing-5b54b7fbb7-tc5vb\" (UID: \"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:34.869854 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:34.869825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:35.065982 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:35.065958    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"]
Sep 08 13:40:35.067762 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:40:35.067735    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6314af19_a498_4ab0_8110_b5156bc17d3f.slice/crio-912f9bd2e64d05cbf1629fe5f2d32a08ce613f11ed71ec7e86690c41531a08f6 WatchSource:0}: Error finding container 912f9bd2e64d05cbf1629fe5f2d32a08ce613f11ed71ec7e86690c41531a08f6: Status 404 returned error can't find the container with id 912f9bd2e64d05cbf1629fe5f2d32a08ce613f11ed71ec7e86690c41531a08f6
Sep 08 13:40:35.149728 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:35.149705    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"]
Sep 08 13:40:35.152083 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:40:35.152055    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7e94ea_a9f6_4811_b42f_98fd19a7bcd8.slice/crio-3422bad14be4c2f733c513910dcfc00d36d356ca2157fd8fb5f79d1452c4a1a9 WatchSource:0}: Error finding container 3422bad14be4c2f733c513910dcfc00d36d356ca2157fd8fb5f79d1452c4a1a9: Status 404 returned error can't find the container with id 3422bad14be4c2f733c513910dcfc00d36d356ca2157fd8fb5f79d1452c4a1a9
Sep 08 13:40:35.237035 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:35.236999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb" event={"ID":"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8","Type":"ContainerStarted","Data":"3422bad14be4c2f733c513910dcfc00d36d356ca2157fd8fb5f79d1452c4a1a9"}
Sep 08 13:40:35.238271 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:35.238247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt" event={"ID":"6314af19-a498-4ab0-8110-b5156bc17d3f","Type":"ContainerStarted","Data":"912f9bd2e64d05cbf1629fe5f2d32a08ce613f11ed71ec7e86690c41531a08f6"}
Sep 08 13:40:38.254051 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:38.254014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt" event={"ID":"6314af19-a498-4ab0-8110-b5156bc17d3f","Type":"ContainerStarted","Data":"c6a4c9d2a495ae5ed23042104f57ce45a895427c3e74e30ad7a755746b5f5acf"}
Sep 08 13:40:38.254583 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:38.254170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:38.255605 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:38.255574    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb" event={"ID":"8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8","Type":"ContainerStarted","Data":"000402e94c4d84b3e1554ff8e2e7a7dd580678e442443fb38fbc3db14272ea7b"}
Sep 08 13:40:38.255715 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:38.255635    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:38.272580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:38.272534    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt" podStartSLOduration=1.595493133 podStartE2EDuration="4.272516324s" totalImagesPullingTime="2.677023191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:40:38.271835907 +0000 UTC m=+721.588239480" watchObservedRunningTime="2026-09-08 13:40:38.272516324 +0000 UTC m=+721.588919879"
Sep 08 13:40:44.284808 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.284776    2577 generic.go:356] "Generic (PLEG): container finished" podID="0460699c-2133-46e6-b289-148b07e9ce69" containerID="b42ec3b35ba2c0e38017c0ab4759c9436d36b2bfd8e71683b063dfd6ffa2c81c" exitCode=0
Sep 08 13:40:44.285215 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.284843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" event={"ID":"0460699c-2133-46e6-b289-148b07e9ce69","Type":"ContainerDied","Data":"b42ec3b35ba2c0e38017c0ab4759c9436d36b2bfd8e71683b063dfd6ffa2c81c"}
Sep 08 13:40:44.285259 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.285226    2577 scope.go:117] "RemoveContainer" containerID="b42ec3b35ba2c0e38017c0ab4759c9436d36b2bfd8e71683b063dfd6ffa2c81c"
Sep 08 13:40:44.286186 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.286162    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:40:44.286247 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.286225    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:40:44.286284 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.286253    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:40:44.309001 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:44.308961    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb" podStartSLOduration=7.713814602 podStartE2EDuration="10.308948054s" totalImagesPullingTime="2.595133452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:40:38.292726218 +0000 UTC m=+721.609129789" watchObservedRunningTime="2026-09-08 13:40:44.308948054 +0000 UTC m=+727.625351608"
Sep 08 13:40:45.294592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:45.294550    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" event={"ID":"0460699c-2133-46e6-b289-148b07e9ce69","Type":"ContainerStarted","Data":"2710f7e08a08ed0da40f157e7193f4c045f8f0489911c37c71f1936f28d9681b"}
Sep 08 13:40:45.299040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:45.299010    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:40:45.300005 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:45.299979    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vtl2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 13:40:45.300200 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:45.300177    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podUID="0460699c-2133-46e6-b289-148b07e9ce69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 13:40:46.305746 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:46.305706    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vtl2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 13:40:46.306226 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:46.305767    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podUID="0460699c-2133-46e6-b289-148b07e9ce69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 13:40:47.299333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:47.299287    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vtl2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 13:40:47.299583 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:47.299382    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podUID="0460699c-2133-46e6-b289-148b07e9ce69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 13:40:48.298962 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:48.298916    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vtl2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 13:40:48.299484 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:48.298984    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podUID="0460699c-2133-46e6-b289-148b07e9ce69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 13:40:49.262292 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:49.262264    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-tc5vb"
Sep 08 13:40:49.262448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:49.262323    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-cx6dt"
Sep 08 13:40:49.306466 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:49.306428    2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-vtl2w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 08 13:40:49.306992 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:49.306961    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w" podUID="0460699c-2133-46e6-b289-148b07e9ce69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 08 13:40:49.809537 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:49.809502    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:40:50.303172 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:50.303142    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:40:50.325588 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:40:50.325562    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-vtl2w"
Sep 08 13:41:56.656997 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.656959    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"]
Sep 08 13:41:56.660709 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.660682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.663484 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.663461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 13:41:56.663622 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.663503    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ws2qv\""
Sep 08 13:41:56.664004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.663988    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 13:41:56.664101 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.664044    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 08 13:41:56.671992 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.671969    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"]
Sep 08 13:41:56.732820 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.732788    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.732820 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.732825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f0c547-a747-4a48-b3cc-19545f9f4786-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.733030 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.732878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gdjn\" (UniqueName: \"kubernetes.io/projected/d6f0c547-a747-4a48-b3cc-19545f9f4786-kube-api-access-4gdjn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.733030 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.732922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.733112 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.733043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.733112 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.733079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833680 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f0c547-a747-4a48-b3cc-19545f9f4786-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gdjn\" (UniqueName: \"kubernetes.io/projected/d6f0c547-a747-4a48-b3cc-19545f9f4786-kube-api-access-4gdjn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.833865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.833854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.834186 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.834164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.834274 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.834248    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.834321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.834276    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.836052 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.836036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6f0c547-a747-4a48-b3cc-19545f9f4786-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.836446 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.836427    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f0c547-a747-4a48-b3cc-19545f9f4786-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.841916 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.841894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gdjn\" (UniqueName: \"kubernetes.io/projected/d6f0c547-a747-4a48-b3cc-19545f9f4786-kube-api-access-4gdjn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8\" (UID: \"d6f0c547-a747-4a48-b3cc-19545f9f4786\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:56.971333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:56.971248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:41:57.244777 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:57.244566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"]
Sep 08 13:41:57.247596 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:41:57.247571    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f0c547_a747_4a48_b3cc_19545f9f4786.slice/crio-1667e7cd436a12accbf8b4ba77696dda730ec8ee4b4e497e62fe0951db5b5834 WatchSource:0}: Error finding container 1667e7cd436a12accbf8b4ba77696dda730ec8ee4b4e497e62fe0951db5b5834: Status 404 returned error can't find the container with id 1667e7cd436a12accbf8b4ba77696dda730ec8ee4b4e497e62fe0951db5b5834
Sep 08 13:41:57.249213 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:57.249196    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:41:57.639457 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:57.639419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8" event={"ID":"d6f0c547-a747-4a48-b3cc-19545f9f4786","Type":"ContainerStarted","Data":"1667e7cd436a12accbf8b4ba77696dda730ec8ee4b4e497e62fe0951db5b5834"}
Sep 08 13:41:58.645994 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:58.645956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8" event={"ID":"d6f0c547-a747-4a48-b3cc-19545f9f4786","Type":"ContainerStarted","Data":"aeb4534d6bbbef54a94270645383e494dfdf8c5409fa8fd0a22258933e070029"}
Sep 08 13:41:58.665740 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:41:58.665687    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8" podStartSLOduration=1.628499675 podStartE2EDuration="2.665671442s" totalImagesPullingTime="1.037171767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:41:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:41:58.663488838 +0000 UTC m=+801.979892402" watchObservedRunningTime="2026-09-08 13:41:58.665671442 +0000 UTC m=+801.982074994"
Sep 08 13:42:02.237469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.237435    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"]
Sep 08 13:42:02.241508 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.241487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.243886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.243858    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 08 13:42:02.249904 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.249367    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"]
Sep 08 13:42:02.382508 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.382682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.382682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.382682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382599    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.382682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pt9c\" (UniqueName: \"kubernetes.io/projected/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-kube-api-access-8pt9c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.382825 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.382694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.483725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.483691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pt9c\" (UniqueName: \"kubernetes.io/projected/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-kube-api-access-8pt9c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.483725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.483728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.483935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.483851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.483935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.483891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.484045 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.483964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.484045 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.484009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.484267 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.484243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.484329 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.484294    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.484404 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.484337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.486015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.485990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.486298 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.486281    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.491972 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.491928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pt9c\" (UniqueName: \"kubernetes.io/projected/d001b502-050b-4a2b-b44a-2a0b4fb6cc36-kube-api-access-8pt9c\") pod \"e2e-trlp-test-simulated-kserve-598466548d-78r5h\" (UID: \"d001b502-050b-4a2b-b44a-2a0b4fb6cc36\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.554215 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.554180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:02.825389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:02.825364    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"]
Sep 08 13:42:02.827828 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:42:02.827800    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd001b502_050b_4a2b_b44a_2a0b4fb6cc36.slice/crio-7b2ee8cd74ef1886118c14865e3ff0b6998d5801213b88d41dcb01c740c7711d WatchSource:0}: Error finding container 7b2ee8cd74ef1886118c14865e3ff0b6998d5801213b88d41dcb01c740c7711d: Status 404 returned error can't find the container with id 7b2ee8cd74ef1886118c14865e3ff0b6998d5801213b88d41dcb01c740c7711d
Sep 08 13:42:03.669602 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:03.669567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h" event={"ID":"d001b502-050b-4a2b-b44a-2a0b4fb6cc36","Type":"ContainerStarted","Data":"a8f930d394031865a2448f219fd074f1d2d804fe7fbde54ce78ef6e2579049ac"}
Sep 08 13:42:03.669980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:03.669625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h" event={"ID":"d001b502-050b-4a2b-b44a-2a0b4fb6cc36","Type":"ContainerStarted","Data":"7b2ee8cd74ef1886118c14865e3ff0b6998d5801213b88d41dcb01c740c7711d"}
Sep 08 13:42:03.690506 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:03.690436    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h" podStartSLOduration=1.4582110400000001 podStartE2EDuration="1.690417983s" totalImagesPullingTime="232.206943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:42:03.688887298 +0000 UTC m=+807.005290870" watchObservedRunningTime="2026-09-08 13:42:03.690417983 +0000 UTC m=+807.006821536"
Sep 08 13:42:06.971675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:06.971632    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:42:06.972076 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:06.971713    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:42:06.984365 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:06.984312    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:42:07.699095 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:07.699062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8"
Sep 08 13:42:12.554927 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:12.554876    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:12.555372 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:12.554962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:12.567705 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:12.567682    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:12.722872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:12.722844    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-78r5h"
Sep 08 13:42:14.436864 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.436824    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"]
Sep 08 13:42:14.440696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.440675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.443228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.443204    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 08 13:42:14.453420 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.453393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"]
Sep 08 13:42:14.597662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67dtd\" (UniqueName: \"kubernetes.io/projected/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-kube-api-access-67dtd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.597848 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.597848 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.597848 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.597848 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.598032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.597910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.698704 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.698704 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67dtd\" (UniqueName: \"kubernetes.io/projected/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-kube-api-access-67dtd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.698915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.698915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.698915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.699037 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.698957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.699176 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.699159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.699321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.699293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.699453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.699407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.701277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.701257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.701472 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.701456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.706167 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.706138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67dtd\" (UniqueName: \"kubernetes.io/projected/51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4-kube-api-access-67dtd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-fvxfg\" (UID: \"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:14.753556 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:14.753523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:15.028167 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:15.028142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"]
Sep 08 13:42:15.030088 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:42:15.030053    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fed2b7_b88f_4ac4_aa49_4c55f3e1edb4.slice/crio-0e47679c70ab00f0793607891ed5b5a40f7aa9aca558a539b33cbe4f3f20df80 WatchSource:0}: Error finding container 0e47679c70ab00f0793607891ed5b5a40f7aa9aca558a539b33cbe4f3f20df80: Status 404 returned error can't find the container with id 0e47679c70ab00f0793607891ed5b5a40f7aa9aca558a539b33cbe4f3f20df80
Sep 08 13:42:15.728010 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:15.727977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg" event={"ID":"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4","Type":"ContainerStarted","Data":"d926deaae4ff76a7ab5bdad63f84608dc7be2dd41d411c1c36b407063449d9ea"}
Sep 08 13:42:15.728429 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:15.728029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg" event={"ID":"51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4","Type":"ContainerStarted","Data":"0e47679c70ab00f0793607891ed5b5a40f7aa9aca558a539b33cbe4f3f20df80"}
Sep 08 13:42:15.746386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:15.746211    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg" podStartSLOduration=1.516185852 podStartE2EDuration="1.746193193s" totalImagesPullingTime="230.007341ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:42:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:42:15.745847182 +0000 UTC m=+819.062250756" watchObservedRunningTime="2026-09-08 13:42:15.746193193 +0000 UTC m=+819.062596748"
Sep 08 13:42:24.754579 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:24.754539    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:24.755007 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:24.754632    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:24.767080 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:24.767038    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:42:24.781606 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:42:24.781579    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-fvxfg"
Sep 08 13:43:37.170064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:43:37.170031    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:43:37.175552 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:43:37.175530    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:44:05.310307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.310267    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:44:05.312667 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.312648    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:44:05.312805 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.312781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.314790 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.314770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.315623 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.315607    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-2d1742\""
Sep 08 13:44:05.316840 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.316824    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-2d1742-dockercfg-72j9c\""
Sep 08 13:44:05.333087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.333069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:44:05.343762 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.343738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:44:05.446114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.446076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxfqd\" (UniqueName: \"kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.446314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.446176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.446314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.446207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2pv9\" (UniqueName: \"kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.446314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.446246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.547064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxfqd\" (UniqueName: \"kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.547231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.547231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2pv9\" (UniqueName: \"kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.547231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.548010 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.548010 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.547946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.557065 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.557038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2pv9\" (UniqueName: \"kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9\") pod \"payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.558086 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.558066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxfqd\" (UniqueName: \"kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd\") pod \"payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") " pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.624243 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.624163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:05.630652 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.630629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:05.902631 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.902591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:44:05.904446 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:05.904410    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa5c8a0_7d0e_4652_96b7_192e025a7a15.slice/crio-deb9f723993d4a8348b585f0d393d3280a823dcb8bf7f4248bdf74904d2029ff WatchSource:0}: Error finding container deb9f723993d4a8348b585f0d393d3280a823dcb8bf7f4248bdf74904d2029ff: Status 404 returned error can't find the container with id deb9f723993d4a8348b585f0d393d3280a823dcb8bf7f4248bdf74904d2029ff
Sep 08 13:44:05.919681 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:05.919661    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:44:05.922255 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:05.922225    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ad22d5_a6ba_460c_b779_ba2b39328fb8.slice/crio-8ce378b46fe06c46b802d6e18932872fb59ac784a78abfa30caae60ca5975e22 WatchSource:0}: Error finding container 8ce378b46fe06c46b802d6e18932872fb59ac784a78abfa30caae60ca5975e22: Status 404 returned error can't find the container with id 8ce378b46fe06c46b802d6e18932872fb59ac784a78abfa30caae60ca5975e22
Sep 08 13:44:06.259217 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.259121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" event={"ID":"2fa5c8a0-7d0e-4652-96b7-192e025a7a15","Type":"ContainerStarted","Data":"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"}
Sep 08 13:44:06.259399 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.259274    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:06.259399 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.259320    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" event={"ID":"2fa5c8a0-7d0e-4652-96b7-192e025a7a15","Type":"ContainerStarted","Data":"deb9f723993d4a8348b585f0d393d3280a823dcb8bf7f4248bdf74904d2029ff"}
Sep 08 13:44:06.260700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.260677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" event={"ID":"f4ad22d5-a6ba-460c-b779-ba2b39328fb8","Type":"ContainerStarted","Data":"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"}
Sep 08 13:44:06.260819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.260712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" event={"ID":"f4ad22d5-a6ba-460c-b779-ba2b39328fb8","Type":"ContainerStarted","Data":"8ce378b46fe06c46b802d6e18932872fb59ac784a78abfa30caae60ca5975e22"}
Sep 08 13:44:06.260819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.260741    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:06.280306 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.280261    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" podStartSLOduration=1.280242803 podStartE2EDuration="1.280242803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:06.278581517 +0000 UTC m=+929.594985100" watchObservedRunningTime="2026-09-08 13:44:06.280242803 +0000 UTC m=+929.596646357"
Sep 08 13:44:06.302597 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:06.302544    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" podStartSLOduration=1.302523518 podStartE2EDuration="1.302523518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:06.299744086 +0000 UTC m=+929.616147664" watchObservedRunningTime="2026-09-08 13:44:06.302523518 +0000 UTC m=+929.618927074"
Sep 08 13:44:13.309794 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.309755    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:13.314932 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.314908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.319717 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.319696    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0b415db2-dockercfg-92zkr\""
Sep 08 13:44:13.320223 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.320204    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0b415db2\""
Sep 08 13:44:13.338465 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.338445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:13.420135 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.420101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5blws\" (UniqueName: \"kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.420333 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.420154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.521671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.521625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5blws\" (UniqueName: \"kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.521861 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.521752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.522501 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.522483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.530793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.530769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5blws\" (UniqueName: \"kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws\") pod \"payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.626866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.626778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:13.905387 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:13.905363    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:13.907975 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:13.907943    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c0b51a_2c87_4809_aa8b_ca1dbdea02e5.slice/crio-a9e127810aca85c302cdff652688eb3cd6758971f6a3e9b637d62e585fbaf93e WatchSource:0}: Error finding container a9e127810aca85c302cdff652688eb3cd6758971f6a3e9b637d62e585fbaf93e: Status 404 returned error can't find the container with id a9e127810aca85c302cdff652688eb3cd6758971f6a3e9b637d62e585fbaf93e
Sep 08 13:44:14.304274 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.304174    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:44:14.307398 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.307370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" event={"ID":"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5","Type":"ContainerStarted","Data":"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"}
Sep 08 13:44:14.307519 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.307416    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" event={"ID":"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5","Type":"ContainerStarted","Data":"a9e127810aca85c302cdff652688eb3cd6758971f6a3e9b637d62e585fbaf93e"}
Sep 08 13:44:14.307519 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.307426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.307519 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.307445    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:14.310675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.310652    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-b719d2\""
Sep 08 13:44:14.323052 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.323026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:44:14.335655 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.335632    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:44:14.338565 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.338550    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.340856 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.340837    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-b719d2-dockercfg-gqrm4\""
Sep 08 13:44:14.343104 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.343066    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" podStartSLOduration=1.343053294 podStartE2EDuration="1.343053294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:14.341505566 +0000 UTC m=+937.657909119" watchObservedRunningTime="2026-09-08 13:44:14.343053294 +0000 UTC m=+937.659456846"
Sep 08 13:44:14.351106 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.351082    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:44:14.432446 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.432415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.432928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.432826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.432928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.432878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p4tf\" (UniqueName: \"kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.432928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.432910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69j4f\" (UniqueName: \"kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.534013 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.533983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.534183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.534070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.534183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.534090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p4tf\" (UniqueName: \"kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.534183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.534109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69j4f\" (UniqueName: \"kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.534784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.534762    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.534840 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.534822    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.542151 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.542123    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p4tf\" (UniqueName: \"kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf\") pod \"payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") " pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.542260 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.542214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69j4f\" (UniqueName: \"kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f\") pod \"payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.618157 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.618082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:14.652524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.652488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:14.891143 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.891115    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:44:14.892156 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:14.892130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03a28fcf_453e_4393_927f_736cfd3d0b1f.slice/crio-bf0411f37ad7dbb3e7d0143ffeb52c8e16d9c12198c62996844b8412ed97a079 WatchSource:0}: Error finding container bf0411f37ad7dbb3e7d0143ffeb52c8e16d9c12198c62996844b8412ed97a079: Status 404 returned error can't find the container with id bf0411f37ad7dbb3e7d0143ffeb52c8e16d9c12198c62996844b8412ed97a079
Sep 08 13:44:14.940903 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:14.940829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:44:14.943552 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:14.943527    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10d2652_72ea_4ed7_b707_c55fd76a119f.slice/crio-a3b6d8e7cbe7a0be0a08218c10175ff90bcf708c96aae0247b8b13c35a7a1636 WatchSource:0}: Error finding container a3b6d8e7cbe7a0be0a08218c10175ff90bcf708c96aae0247b8b13c35a7a1636: Status 404 returned error can't find the container with id a3b6d8e7cbe7a0be0a08218c10175ff90bcf708c96aae0247b8b13c35a7a1636
Sep 08 13:44:15.311784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.311688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" event={"ID":"03a28fcf-453e-4393-927f-736cfd3d0b1f","Type":"ContainerStarted","Data":"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"}
Sep 08 13:44:15.311784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.311757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:15.311784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.311783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" event={"ID":"03a28fcf-453e-4393-927f-736cfd3d0b1f","Type":"ContainerStarted","Data":"bf0411f37ad7dbb3e7d0143ffeb52c8e16d9c12198c62996844b8412ed97a079"}
Sep 08 13:44:15.313532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.313505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" event={"ID":"e10d2652-72ea-4ed7-b707-c55fd76a119f","Type":"ContainerStarted","Data":"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"}
Sep 08 13:44:15.313656 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.313554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" event={"ID":"e10d2652-72ea-4ed7-b707-c55fd76a119f","Type":"ContainerStarted","Data":"a3b6d8e7cbe7a0be0a08218c10175ff90bcf708c96aae0247b8b13c35a7a1636"}
Sep 08 13:44:15.313656 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.313594    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:15.334055 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.334002    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" podStartSLOduration=1.333983077 podStartE2EDuration="1.333983077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:15.331478828 +0000 UTC m=+938.647882403" watchObservedRunningTime="2026-09-08 13:44:15.333983077 +0000 UTC m=+938.650386668"
Sep 08 13:44:15.364110 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:15.364052    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" podStartSLOduration=1.364004542 podStartE2EDuration="1.364004542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:15.362135899 +0000 UTC m=+938.678539486" watchObservedRunningTime="2026-09-08 13:44:15.364004542 +0000 UTC m=+938.680408098"
Sep 08 13:44:17.268018 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:17.267989    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:44:17.268488 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:17.268151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:44:25.315301 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:25.315269    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:26.321161 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:26.321128    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:44:26.321554 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:26.321310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:44:35.120210 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.120176    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:44:35.123675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.123659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.128481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.128461    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a7ef4b\""
Sep 08 13:44:35.141550 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.141532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:44:35.152309 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.152286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:44:35.155897 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.155878    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.164205 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.164185    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a7ef4b-dockercfg-sdlx9\""
Sep 08 13:44:35.172086 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.172064    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:44:35.244057 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.244017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45hmg\" (UniqueName: \"kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.244286 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.244148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.244286 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.244240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.244494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.244469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drw6b\" (UniqueName: \"kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.345860 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.345823    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45hmg\" (UniqueName: \"kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.346077 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.345883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.346077 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.346064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.346201 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.346121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drw6b\" (UniqueName: \"kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.346823 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.346801    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.346871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.346826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.355867 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.355830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drw6b\" (UniqueName: \"kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b\") pod \"payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.356154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.356133    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45hmg\" (UniqueName: \"kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg\") pod \"payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.434600 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.434566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:35.467053 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.467022    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:35.715262 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.715237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:44:35.717649 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:35.717623    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c41050c_d4b4_45b2_b627_223a794f78fc.slice/crio-4a48b090095eb70a3c9fc001352348e701d31cc4fc7fbea44239ccdd942029af WatchSource:0}: Error finding container 4a48b090095eb70a3c9fc001352348e701d31cc4fc7fbea44239ccdd942029af: Status 404 returned error can't find the container with id 4a48b090095eb70a3c9fc001352348e701d31cc4fc7fbea44239ccdd942029af
Sep 08 13:44:35.758143 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:35.758114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:44:35.762894 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:44:35.762864    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbe5639_89e2_4ae1_8261_2e8b4cfb91ef.slice/crio-f47ab8812de85799d08532d08ca8bb7f20f917f753e3c67ac847a3663bb2b326 WatchSource:0}: Error finding container f47ab8812de85799d08532d08ca8bb7f20f917f753e3c67ac847a3663bb2b326: Status 404 returned error can't find the container with id f47ab8812de85799d08532d08ca8bb7f20f917f753e3c67ac847a3663bb2b326
Sep 08 13:44:36.425292 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.425249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" event={"ID":"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef","Type":"ContainerStarted","Data":"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"}
Sep 08 13:44:36.425841 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.425364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" event={"ID":"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef","Type":"ContainerStarted","Data":"f47ab8812de85799d08532d08ca8bb7f20f917f753e3c67ac847a3663bb2b326"}
Sep 08 13:44:36.425841 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.425499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:36.427217 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.427194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" event={"ID":"6c41050c-d4b4-45b2-b627-223a794f78fc","Type":"ContainerStarted","Data":"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"}
Sep 08 13:44:36.427378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.427236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" event={"ID":"6c41050c-d4b4-45b2-b627-223a794f78fc","Type":"ContainerStarted","Data":"4a48b090095eb70a3c9fc001352348e701d31cc4fc7fbea44239ccdd942029af"}
Sep 08 13:44:36.427378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.427264    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:36.448158 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.448106    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" podStartSLOduration=1.448092801 podStartE2EDuration="1.448092801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:36.446329605 +0000 UTC m=+959.762733182" watchObservedRunningTime="2026-09-08 13:44:36.448092801 +0000 UTC m=+959.764496354"
Sep 08 13:44:36.464494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:36.464451    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" podStartSLOduration=1.46443323 podStartE2EDuration="1.46443323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:36.463790099 +0000 UTC m=+959.780193674" watchObservedRunningTime="2026-09-08 13:44:36.46443323 +0000 UTC m=+959.780836783"
Sep 08 13:44:47.440431 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:47.440390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:44:47.440856 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:47.440478    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:44:53.431974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.431938    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:53.433471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.433267    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" podUID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" containerName="payload-pre-processing" containerID="cri-o://262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a" gracePeriod=30
Sep 08 13:44:53.687196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.687144    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:53.834129 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.834096    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume\") pod \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") "
Sep 08 13:44:53.834327 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.834166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5blws\" (UniqueName: \"kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws\") pod \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\" (UID: \"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5\") "
Sep 08 13:44:53.834524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.834498    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" (UID: "e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:44:53.836513 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.836490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws" (OuterVolumeSpecName: "kube-api-access-5blws") pod "e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" (UID: "e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5"). InnerVolumeSpecName "kube-api-access-5blws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:44:53.935115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.935073    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:44:53.935115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:53.935110    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5blws\" (UniqueName: \"kubernetes.io/projected/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5-kube-api-access-5blws\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:44:54.527177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.527146    2577 generic.go:356] "Generic (PLEG): container finished" podID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" containerID="262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a" exitCode=0
Sep 08 13:44:54.527673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.527192    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"
Sep 08 13:44:54.527673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.527201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" event={"ID":"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5","Type":"ContainerDied","Data":"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"}
Sep 08 13:44:54.527673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.527257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv" event={"ID":"e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5","Type":"ContainerDied","Data":"a9e127810aca85c302cdff652688eb3cd6758971f6a3e9b637d62e585fbaf93e"}
Sep 08 13:44:54.527673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.527298    2577 scope.go:117] "RemoveContainer" containerID="262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"
Sep 08 13:44:54.537498 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.537478    2577 scope.go:117] "RemoveContainer" containerID="262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"
Sep 08 13:44:54.537801 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:44:54.537781    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a\": container with ID starting with 262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a not found: ID does not exist" containerID="262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"
Sep 08 13:44:54.537854 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.537816    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a"} err="failed to get container status \"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a\": rpc error: code = NotFound desc = could not find container \"262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a\": container with ID starting with 262de1a8e39892ae6111acf698359b2150c3b123fa7b9e43bfd89a2d12346e0a not found: ID does not exist"
Sep 08 13:44:54.554591 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.554570    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:54.559577 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:54.559558    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0b415db2-54c6dbf446-fskdv"]
Sep 08 13:44:55.212756 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:44:55.212727    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" path="/var/lib/kubelet/pods/e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5/volumes"
Sep 08 13:45:00.143411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.143282    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"]
Sep 08 13:45:00.143846 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.143738    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" containerName="payload-pre-processing"
Sep 08 13:45:00.143846 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.143749    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" containerName="payload-pre-processing"
Sep 08 13:45:00.143846 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.143811    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5c0b51a-2c87-4809-aa8b-ca1dbdea02e5" containerName="payload-pre-processing"
Sep 08 13:45:00.148066 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.148046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"]
Sep 08 13:45:00.148191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.148178    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"
Sep 08 13:45:00.150305 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.150285    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:45:00.151580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.151562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"
Sep 08 13:45:00.157779 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.157757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"]
Sep 08 13:45:00.160221 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.160191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"
Sep 08 13:45:00.160675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.160622    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"]
Sep 08 13:45:00.169712 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.169694    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"
Sep 08 13:45:00.444682 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.444640    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"]
Sep 08 13:45:00.446202 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:45:00.446149    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f0c29d_817e_4e10_9f78_5b6cf6d59fcf.slice/crio-dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251 WatchSource:0}: Error finding container dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251: Status 404 returned error can't find the container with id dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251
Sep 08 13:45:00.459475 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.459449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"]
Sep 08 13:45:00.462772 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:45:00.462750    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148dfbb4_98cd_43ef_820b_f8b2e1ea2e75.slice/crio-eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123 WatchSource:0}: Error finding container eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123: Status 404 returned error can't find the container with id eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123
Sep 08 13:45:00.566495 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.566464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29" event={"ID":"148dfbb4-98cd-43ef-820b-f8b2e1ea2e75","Type":"ContainerStarted","Data":"eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123"}
Sep 08 13:45:00.568278 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:00.568251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg" event={"ID":"50f0c29d-817e-4e10-9f78-5b6cf6d59fcf","Type":"ContainerStarted","Data":"dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251"}
Sep 08 13:45:03.589310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:03.589268    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29" event={"ID":"148dfbb4-98cd-43ef-820b-f8b2e1ea2e75","Type":"ContainerStarted","Data":"84440671db2104666dc5c1319a58217341d9e6b892b06043925573b5ce8fa380"}
Sep 08 13:45:03.591141 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:03.591100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg" event={"ID":"50f0c29d-817e-4e10-9f78-5b6cf6d59fcf","Type":"ContainerStarted","Data":"79bfed6fb6420d107ec90b107a5b6bfd71e317a7dffd0956863e145fc70e3807"}
Sep 08 13:45:03.605088 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:03.605043    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29" podStartSLOduration=1.349279939 podStartE2EDuration="3.605027395s" totalImagesPullingTime="2.255747456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:45:03.604818927 +0000 UTC m=+986.921222498" watchObservedRunningTime="2026-09-08 13:45:03.605027395 +0000 UTC m=+986.921430946"
Sep 08 13:45:03.621057 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:03.621018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg" podStartSLOduration=1.353258187 podStartE2EDuration="3.621005304s" totalImagesPullingTime="2.267747117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:45:03.619899757 +0000 UTC m=+986.936303328" watchObservedRunningTime="2026-09-08 13:45:03.621005304 +0000 UTC m=+986.937408855"
Sep 08 13:45:04.598453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:04.598423    2577 generic.go:356] "Generic (PLEG): container finished" podID="50f0c29d-817e-4e10-9f78-5b6cf6d59fcf" containerID="79bfed6fb6420d107ec90b107a5b6bfd71e317a7dffd0956863e145fc70e3807" exitCode=0
Sep 08 13:45:04.598871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:04.598517    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg" event={"ID":"50f0c29d-817e-4e10-9f78-5b6cf6d59fcf","Type":"ContainerDied","Data":"79bfed6fb6420d107ec90b107a5b6bfd71e317a7dffd0956863e145fc70e3807"}
Sep 08 13:45:04.600037 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:04.600016    2577 generic.go:356] "Generic (PLEG): container finished" podID="148dfbb4-98cd-43ef-820b-f8b2e1ea2e75" containerID="84440671db2104666dc5c1319a58217341d9e6b892b06043925573b5ce8fa380" exitCode=0
Sep 08 13:45:04.600141 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:04.600080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29" event={"ID":"148dfbb4-98cd-43ef-820b-f8b2e1ea2e75","Type":"ContainerDied","Data":"84440671db2104666dc5c1319a58217341d9e6b892b06043925573b5ce8fa380"}
Sep 08 13:45:05.746821 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:05.746797    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"
Sep 08 13:45:05.768471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:05.768442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"
Sep 08 13:45:06.612192 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.612157    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"
Sep 08 13:45:06.612425 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.612187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29" event={"ID":"148dfbb4-98cd-43ef-820b-f8b2e1ea2e75","Type":"ContainerDied","Data":"eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123"}
Sep 08 13:45:06.612425 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.612242    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb3271c9400301c90e48f612af2ed13f6f6cbe0cc7eb2226ea989197bcf4c123"
Sep 08 13:45:06.617089 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.617060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg" event={"ID":"50f0c29d-817e-4e10-9f78-5b6cf6d59fcf","Type":"ContainerDied","Data":"dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251"}
Sep 08 13:45:06.617216 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.617108    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814585-xm6tg"
Sep 08 13:45:06.617216 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:45:06.617111    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd7f86e177eff41b7af0a839522b0b1cd66b253b1270ac11eaba3f9b5a3c0251"
Sep 08 13:46:16.030388 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.030336    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:46:16.032975 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.030579    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerName="payload-processing" containerID="cri-o://7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247" gracePeriod=30
Sep 08 13:46:16.038841 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.038817    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:46:16.039060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.039036    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerName="payload-pre-processing" containerID="cri-o://07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117" gracePeriod=30
Sep 08 13:46:16.337794 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.337769    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:46:16.340636 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.340617    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:46:16.459957 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.459923    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume\") pod \"03a28fcf-453e-4393-927f-736cfd3d0b1f\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") "
Sep 08 13:46:16.460144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.459990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume\") pod \"e10d2652-72ea-4ed7-b707-c55fd76a119f\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") "
Sep 08 13:46:16.460144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460033    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69j4f\" (UniqueName: \"kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f\") pod \"03a28fcf-453e-4393-927f-736cfd3d0b1f\" (UID: \"03a28fcf-453e-4393-927f-736cfd3d0b1f\") "
Sep 08 13:46:16.460144 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460083    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p4tf\" (UniqueName: \"kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf\") pod \"e10d2652-72ea-4ed7-b707-c55fd76a119f\" (UID: \"e10d2652-72ea-4ed7-b707-c55fd76a119f\") "
Sep 08 13:46:16.460336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460307    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03a28fcf-453e-4393-927f-736cfd3d0b1f" (UID: "03a28fcf-453e-4393-927f-736cfd3d0b1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:16.460428 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460317    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e10d2652-72ea-4ed7-b707-c55fd76a119f" (UID: "e10d2652-72ea-4ed7-b707-c55fd76a119f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:16.460467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460447    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03a28fcf-453e-4393-927f-736cfd3d0b1f-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.460500 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.460469    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e10d2652-72ea-4ed7-b707-c55fd76a119f-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.462314 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.462295    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f" (OuterVolumeSpecName: "kube-api-access-69j4f") pod "03a28fcf-453e-4393-927f-736cfd3d0b1f" (UID: "03a28fcf-453e-4393-927f-736cfd3d0b1f"). InnerVolumeSpecName "kube-api-access-69j4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:46:16.462474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.462458    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf" (OuterVolumeSpecName: "kube-api-access-4p4tf") pod "e10d2652-72ea-4ed7-b707-c55fd76a119f" (UID: "e10d2652-72ea-4ed7-b707-c55fd76a119f"). InnerVolumeSpecName "kube-api-access-4p4tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:46:16.561520 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.561439    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69j4f\" (UniqueName: \"kubernetes.io/projected/03a28fcf-453e-4393-927f-736cfd3d0b1f-kube-api-access-69j4f\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.561520 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.561464    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p4tf\" (UniqueName: \"kubernetes.io/projected/e10d2652-72ea-4ed7-b707-c55fd76a119f-kube-api-access-4p4tf\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.987316 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.987274    2577 generic.go:356] "Generic (PLEG): container finished" podID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerID="7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247" exitCode=0
Sep 08 13:46:16.987509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.987329    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"
Sep 08 13:46:16.987509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.987364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" event={"ID":"e10d2652-72ea-4ed7-b707-c55fd76a119f","Type":"ContainerDied","Data":"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"}
Sep 08 13:46:16.987509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.987420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" event={"ID":"e10d2652-72ea-4ed7-b707-c55fd76a119f","Type":"ContainerDied","Data":"a3b6d8e7cbe7a0be0a08218c10175ff90bcf708c96aae0247b8b13c35a7a1636"}
Sep 08 13:46:16.987509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.987442    2577 scope.go:117] "RemoveContainer" containerID="7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"
Sep 08 13:46:16.988859 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.988836    2577 generic.go:356] "Generic (PLEG): container finished" podID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerID="07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117" exitCode=0
Sep 08 13:46:16.988967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.988879    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"
Sep 08 13:46:16.988967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.988894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" event={"ID":"03a28fcf-453e-4393-927f-736cfd3d0b1f","Type":"ContainerDied","Data":"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"}
Sep 08 13:46:16.988967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.988939    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" event={"ID":"03a28fcf-453e-4393-927f-736cfd3d0b1f","Type":"ContainerDied","Data":"bf0411f37ad7dbb3e7d0143ffeb52c8e16d9c12198c62996844b8412ed97a079"}
Sep 08 13:46:16.998029 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.998012    2577 scope.go:117] "RemoveContainer" containerID="7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"
Sep 08 13:46:16.998390 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:46:16.998281    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247\": container with ID starting with 7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247 not found: ID does not exist" containerID="7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"
Sep 08 13:46:16.998390 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.998304    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247"} err="failed to get container status \"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247\": rpc error: code = NotFound desc = could not find container \"7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247\": container with ID starting with 7f2518e7df25a3761fb459f2eeefcc4e4208df4079c7590fc03d29f8642a1247 not found: ID does not exist"
Sep 08 13:46:16.998390 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:16.998326    2577 scope.go:117] "RemoveContainer" containerID="07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"
Sep 08 13:46:17.006607 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.006592    2577 scope.go:117] "RemoveContainer" containerID="07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"
Sep 08 13:46:17.006834 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:46:17.006813    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117\": container with ID starting with 07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117 not found: ID does not exist" containerID="07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"
Sep 08 13:46:17.006878 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.006841    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117"} err="failed to get container status \"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117\": rpc error: code = NotFound desc = could not find container \"07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117\": container with ID starting with 07d2ec817e678794b2558fceebcb23b481a46892101392bd2586b1a1596cc117 not found: ID does not exist"
Sep 08 13:46:17.011169 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.011147    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:46:17.015288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.015269    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj"]
Sep 08 13:46:17.026831 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.026810    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:46:17.030462 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.030446    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd"]
Sep 08 13:46:17.213740 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.213708    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" path="/var/lib/kubelet/pods/03a28fcf-453e-4393-927f-736cfd3d0b1f/volumes"
Sep 08 13:46:17.214066 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.214052    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" path="/var/lib/kubelet/pods/e10d2652-72ea-4ed7-b707-c55fd76a119f/volumes"
Sep 08 13:46:17.320039 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.319955    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-autores-b719d2-b64fbfd87-qvfwj" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerName="payload-processing" probeResult="failure" output="dial tcp 10.134.0.39:9004: i/o timeout"
Sep 08 13:46:17.320039 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:17.319966    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-autores-b719d2-9bfcb6694-lkffd" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.38:9004: i/o timeout"
Sep 08 13:46:33.861220 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861178    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"]
Sep 08 13:46:33.861808 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861785    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerName="payload-pre-processing"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861810    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerName="payload-pre-processing"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861821    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerName="payload-processing"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861829    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerName="payload-processing"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861853    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f0c29d-817e-4e10-9f78-5b6cf6d59fcf" containerName="cleanup"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861861    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f0c29d-817e-4e10-9f78-5b6cf6d59fcf" containerName="cleanup"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861879    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="148dfbb4-98cd-43ef-820b-f8b2e1ea2e75" containerName="cleanup"
Sep 08 13:46:33.861898 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861887    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="148dfbb4-98cd-43ef-820b-f8b2e1ea2e75" containerName="cleanup"
Sep 08 13:46:33.862233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.861994    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10d2652-72ea-4ed7-b707-c55fd76a119f" containerName="payload-processing"
Sep 08 13:46:33.862233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.862010    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a28fcf-453e-4393-927f-736cfd3d0b1f" containerName="payload-pre-processing"
Sep 08 13:46:33.862233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.862023    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f0c29d-817e-4e10-9f78-5b6cf6d59fcf" containerName="cleanup"
Sep 08 13:46:33.862233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.862032    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="148dfbb4-98cd-43ef-820b-f8b2e1ea2e75" containerName="cleanup"
Sep 08 13:46:33.865186 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.865166    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:33.870357 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.870322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:46:33.879160 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.879138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"]
Sep 08 13:46:33.915875 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:33.915835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2781f063-7fxbv\" (UID: \"9770f944-cd26-47e8-8b04-3c16d207f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:34.016660 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:34.016618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2781f063-7fxbv\" (UID: \"9770f944-cd26-47e8-8b04-3c16d207f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:34.017277 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:34.017254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2781f063-7fxbv\" (UID: \"9770f944-cd26-47e8-8b04-3c16d207f07c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:34.175694 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:34.175655    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:34.449720 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:34.449695    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"]
Sep 08 13:46:34.451671 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:46:34.451645    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9770f944_cd26_47e8_8b04_3c16d207f07c.slice/crio-374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9 WatchSource:0}: Error finding container 374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9: Status 404 returned error can't find the container with id 374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9
Sep 08 13:46:35.083725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:35.083688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv" event={"ID":"9770f944-cd26-47e8-8b04-3c16d207f07c","Type":"ContainerStarted","Data":"8bea503dcab01098464c1faf82ae697059df6f15e9bcc9825fbad0c495424730"}
Sep 08 13:46:35.084119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:35.083741    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv" event={"ID":"9770f944-cd26-47e8-8b04-3c16d207f07c","Type":"ContainerStarted","Data":"374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9"}
Sep 08 13:46:35.099854 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:35.099800    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv" podStartSLOduration=2.099774008 podStartE2EDuration="2.099774008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:46:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:46:35.098275502 +0000 UTC m=+1078.414679084" watchObservedRunningTime="2026-09-08 13:46:35.099774008 +0000 UTC m=+1078.416177561"
Sep 08 13:46:36.090930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:36.090895    2577 generic.go:356] "Generic (PLEG): container finished" podID="9770f944-cd26-47e8-8b04-3c16d207f07c" containerID="8bea503dcab01098464c1faf82ae697059df6f15e9bcc9825fbad0c495424730" exitCode=0
Sep 08 13:46:36.091325 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:36.090943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv" event={"ID":"9770f944-cd26-47e8-8b04-3c16d207f07c","Type":"ContainerDied","Data":"8bea503dcab01098464c1faf82ae697059df6f15e9bcc9825fbad0c495424730"}
Sep 08 13:46:37.239838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:37.239815    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:37.347237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:37.347153    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca\") pod \"9770f944-cd26-47e8-8b04-3c16d207f07c\" (UID: \"9770f944-cd26-47e8-8b04-3c16d207f07c\") "
Sep 08 13:46:37.347571 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:37.347541    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9770f944-cd26-47e8-8b04-3c16d207f07c" (UID: "9770f944-cd26-47e8-8b04-3c16d207f07c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:37.447971 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:37.447945    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9770f944-cd26-47e8-8b04-3c16d207f07c-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:46:38.102851 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:38.102820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv" event={"ID":"9770f944-cd26-47e8-8b04-3c16d207f07c","Type":"ContainerDied","Data":"374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9"}
Sep 08 13:46:38.102851 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:38.102842    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"
Sep 08 13:46:38.103059 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:38.102866    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="374c2a0e1ec7818c78568b1cf64475d320b93db7d2d888a87f0c88e8bd5acbb9"
Sep 08 13:46:40.950180 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:40.950141    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"]
Sep 08 13:46:40.952881 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:40.952854    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2781f063-7fxbv"]
Sep 08 13:46:41.212735 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:46:41.212652    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9770f944-cd26-47e8-8b04-3c16d207f07c" path="/var/lib/kubelet/pods/9770f944-cd26-47e8-8b04-3c16d207f07c/volumes"
Sep 08 13:47:45.790782 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.790737    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"]
Sep 08 13:47:45.791415 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.791187    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9770f944-cd26-47e8-8b04-3c16d207f07c" containerName="revoke-keys"
Sep 08 13:47:45.791415 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.791199    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9770f944-cd26-47e8-8b04-3c16d207f07c" containerName="revoke-keys"
Sep 08 13:47:45.791415 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.791254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9770f944-cd26-47e8-8b04-3c16d207f07c" containerName="revoke-keys"
Sep 08 13:47:45.794449 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.794427    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:45.796690 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.796672    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:47:45.801200 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.801173    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"]
Sep 08 13:47:45.811563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.811537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk\" (UID: \"c53e7980-0463-4551-9430-a1e7da36ebf5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:45.912285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.912249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk\" (UID: \"c53e7980-0463-4551-9430-a1e7da36ebf5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:45.912954 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:45.912937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk\" (UID: \"c53e7980-0463-4551-9430-a1e7da36ebf5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:46.109533 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:46.109436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:46.383032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:46.382995    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"]
Sep 08 13:47:46.385568 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:47:46.385535    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53e7980_0463_4551_9430_a1e7da36ebf5.slice/crio-d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b WatchSource:0}: Error finding container d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b: Status 404 returned error can't find the container with id d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b
Sep 08 13:47:46.465654 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:46.465620    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk" event={"ID":"c53e7980-0463-4551-9430-a1e7da36ebf5","Type":"ContainerStarted","Data":"d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b"}
Sep 08 13:47:47.472842 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:47.472804    2577 generic.go:356] "Generic (PLEG): container finished" podID="c53e7980-0463-4551-9430-a1e7da36ebf5" containerID="1a6a3371b6f9ade436444133685650e188b3c817ca0da3c77513aa0b6c344836" exitCode=0
Sep 08 13:47:47.473269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:47.472888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk" event={"ID":"c53e7980-0463-4551-9430-a1e7da36ebf5","Type":"ContainerDied","Data":"1a6a3371b6f9ade436444133685650e188b3c817ca0da3c77513aa0b6c344836"}
Sep 08 13:47:48.616164 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:48.616139    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:48.636510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:48.636486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca\") pod \"c53e7980-0463-4551-9430-a1e7da36ebf5\" (UID: \"c53e7980-0463-4551-9430-a1e7da36ebf5\") "
Sep 08 13:47:48.636863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:48.636834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c53e7980-0463-4551-9430-a1e7da36ebf5" (UID: "c53e7980-0463-4551-9430-a1e7da36ebf5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:47:48.738128 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:48.738046    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c53e7980-0463-4551-9430-a1e7da36ebf5-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:47:49.484240 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:49.484150    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"
Sep 08 13:47:49.484240 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:49.484163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk" event={"ID":"c53e7980-0463-4551-9430-a1e7da36ebf5","Type":"ContainerDied","Data":"d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b"}
Sep 08 13:47:49.484240 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:49.484212    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6b6857ade050680db8e84f2e878f7d8afa9a6caf22492744d920d7a862c333b"
Sep 08 13:47:56.802147 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:56.802109    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"]
Sep 08 13:47:56.807937 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:56.807908    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-d60b5a-rhlxk"]
Sep 08 13:47:57.212708 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:47:57.212676    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c53e7980-0463-4551-9430-a1e7da36ebf5" path="/var/lib/kubelet/pods/c53e7980-0463-4551-9430-a1e7da36ebf5/volumes"
Sep 08 13:48:34.508289 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.508251    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:48:34.508928 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.508907    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c53e7980-0463-4551-9430-a1e7da36ebf5" containerName="revoke-keys"
Sep 08 13:48:34.508975 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.508934    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c53e7980-0463-4551-9430-a1e7da36ebf5" containerName="revoke-keys"
Sep 08 13:48:34.509064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.509053    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c53e7980-0463-4551-9430-a1e7da36ebf5" containerName="revoke-keys"
Sep 08 13:48:34.512785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.512764    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.515469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.515414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f8154283-dockercfg-6rbzg\""
Sep 08 13:48:34.515469 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.515437    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f8154283\""
Sep 08 13:48:34.524562 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.524536    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:48:34.593246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.593206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxx4j\" (UniqueName: \"kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.593441 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.593329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.694702 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.694662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxx4j\" (UniqueName: \"kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.694872 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.694735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.695411 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.695392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.702571 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.702546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxx4j\" (UniqueName: \"kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j\") pod \"payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:34.828488 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:34.828405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:35.100718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:35.100691    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:48:35.102755 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:48:35.102722    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod044a8c52_0528_41cd_bfab_d93b39be2adb.slice/crio-8e5d9f7439d8b8bc13ed0c6a70bc12a94203afd6a39ad177db1b7f2bbe157bc5 WatchSource:0}: Error finding container 8e5d9f7439d8b8bc13ed0c6a70bc12a94203afd6a39ad177db1b7f2bbe157bc5: Status 404 returned error can't find the container with id 8e5d9f7439d8b8bc13ed0c6a70bc12a94203afd6a39ad177db1b7f2bbe157bc5
Sep 08 13:48:35.724983 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:35.724944    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" event={"ID":"044a8c52-0528-41cd-bfab-d93b39be2adb","Type":"ContainerStarted","Data":"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"}
Sep 08 13:48:35.725567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:35.725000    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" event={"ID":"044a8c52-0528-41cd-bfab-d93b39be2adb","Type":"ContainerStarted","Data":"8e5d9f7439d8b8bc13ed0c6a70bc12a94203afd6a39ad177db1b7f2bbe157bc5"}
Sep 08 13:48:35.725567 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:35.725082    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:35.740980 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:35.740930    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" podStartSLOduration=1.740914815 podStartE2EDuration="1.740914815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:48:35.740280492 +0000 UTC m=+1199.056684068" watchObservedRunningTime="2026-09-08 13:48:35.740914815 +0000 UTC m=+1199.057318387"
Sep 08 13:48:37.205145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:37.205118    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:48:37.208444 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:37.208419    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:48:46.732211 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:46.732179    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:48:47.393658 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.393621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"]
Sep 08 13:48:47.399105 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.399078    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:47.401828 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.401807    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:48:47.406184 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.406155    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"]
Sep 08 13:48:47.512932 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.512891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h\" (UID: \"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:47.613693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.613656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h\" (UID: \"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:47.614571 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.614552    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h\" (UID: \"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:47.711489 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.711396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:47.983653 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:47.983628    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"]
Sep 08 13:48:47.986183 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:48:47.986151    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef244f3c_3fff_4e22_aa3b_dc9b8a01f96a.slice/crio-aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673 WatchSource:0}: Error finding container aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673: Status 404 returned error can't find the container with id aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673
Sep 08 13:48:48.794951 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:48.794916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h" event={"ID":"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a","Type":"ContainerStarted","Data":"79b562c39b56b748dc5f891e7a480fbc8914a96e1e3b4ed0daf9159fce553c3a"}
Sep 08 13:48:48.795087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:48.794976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h" event={"ID":"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a","Type":"ContainerStarted","Data":"aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673"}
Sep 08 13:48:48.811006 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:48.810946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h" podStartSLOduration=1.81092359 podStartE2EDuration="1.81092359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:48:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:48:48.810196403 +0000 UTC m=+1212.126599954" watchObservedRunningTime="2026-09-08 13:48:48.81092359 +0000 UTC m=+1212.127327142"
Sep 08 13:48:49.801621 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:49.801592    2577 generic.go:356] "Generic (PLEG): container finished" podID="ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" containerID="79b562c39b56b748dc5f891e7a480fbc8914a96e1e3b4ed0daf9159fce553c3a" exitCode=0
Sep 08 13:48:49.801988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:49.801687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h" event={"ID":"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a","Type":"ContainerDied","Data":"79b562c39b56b748dc5f891e7a480fbc8914a96e1e3b4ed0daf9159fce553c3a"}
Sep 08 13:48:50.928797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:50.928776    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:51.047385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.047301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca\") pod \"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a\" (UID: \"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a\") "
Sep 08 13:48:51.047704 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.047679    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" (UID: "ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:48:51.148675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.148643    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:48:51.813083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.813054    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"
Sep 08 13:48:51.813083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.813067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h" event={"ID":"ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a","Type":"ContainerDied","Data":"aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673"}
Sep 08 13:48:51.813389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:51.813109    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aba13a60512c837c5b6a72707b279cbd6fd13967f8fc9d9fe7c85e6e1c6b5673"
Sep 08 13:48:58.153985 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.153950    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:48:58.154481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.154206    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" podUID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" containerName="payload-processing" containerID="cri-o://cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c" gracePeriod=30
Sep 08 13:48:58.163059 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.162508    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:48:58.163059 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.162722    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" podUID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" containerName="payload-pre-processing" containerID="cri-o://8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc" gracePeriod=30
Sep 08 13:48:58.200166 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:48:58.200131    2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ad22d5_a6ba_460c_b779_ba2b39328fb8.slice/crio-cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa5c8a0_7d0e_4652_96b7_192e025a7a15.slice/crio-conmon-8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc.scope\": RecentStats: unable to find data in memory cache]"
Sep 08 13:48:58.346073 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.346039    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"]
Sep 08 13:48:58.360278 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.360248    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-2d1742-6qk9h"]
Sep 08 13:48:58.444756 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.444730    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:48:58.447657 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.447630    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:48:58.624212 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624179    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume\") pod \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") "
Sep 08 13:48:58.624405 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2pv9\" (UniqueName: \"kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9\") pod \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") "
Sep 08 13:48:58.624405 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624295    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume\") pod \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\" (UID: \"2fa5c8a0-7d0e-4652-96b7-192e025a7a15\") "
Sep 08 13:48:58.624405 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624375    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxfqd\" (UniqueName: \"kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd\") pod \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\" (UID: \"f4ad22d5-a6ba-460c-b779-ba2b39328fb8\") "
Sep 08 13:48:58.624616 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624593    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4ad22d5-a6ba-460c-b779-ba2b39328fb8" (UID: "f4ad22d5-a6ba-460c-b779-ba2b39328fb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:48:58.624733 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624712    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.624733 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.624724    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fa5c8a0-7d0e-4652-96b7-192e025a7a15" (UID: "2fa5c8a0-7d0e-4652-96b7-192e025a7a15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:48:58.626538 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.626519    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9" (OuterVolumeSpecName: "kube-api-access-l2pv9") pod "2fa5c8a0-7d0e-4652-96b7-192e025a7a15" (UID: "2fa5c8a0-7d0e-4652-96b7-192e025a7a15"). InnerVolumeSpecName "kube-api-access-l2pv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:48:58.626670 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.626653    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd" (OuterVolumeSpecName: "kube-api-access-vxfqd") pod "f4ad22d5-a6ba-460c-b779-ba2b39328fb8" (UID: "f4ad22d5-a6ba-460c-b779-ba2b39328fb8"). InnerVolumeSpecName "kube-api-access-vxfqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:48:58.725671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.725579    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2pv9\" (UniqueName: \"kubernetes.io/projected/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-kube-api-access-l2pv9\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.725671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.725615    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fa5c8a0-7d0e-4652-96b7-192e025a7a15-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.725671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.725626    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxfqd\" (UniqueName: \"kubernetes.io/projected/f4ad22d5-a6ba-460c-b779-ba2b39328fb8-kube-api-access-vxfqd\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.851050 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.851013    2577 generic.go:356] "Generic (PLEG): container finished" podID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" containerID="cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c" exitCode=0
Sep 08 13:48:58.851255 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.851076    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"
Sep 08 13:48:58.851255 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.851100    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" event={"ID":"f4ad22d5-a6ba-460c-b779-ba2b39328fb8","Type":"ContainerDied","Data":"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"}
Sep 08 13:48:58.851255 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.851157    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj" event={"ID":"f4ad22d5-a6ba-460c-b779-ba2b39328fb8","Type":"ContainerDied","Data":"8ce378b46fe06c46b802d6e18932872fb59ac784a78abfa30caae60ca5975e22"}
Sep 08 13:48:58.851255 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.851187    2577 scope.go:117] "RemoveContainer" containerID="cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"
Sep 08 13:48:58.857642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.852899    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" containerID="8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc" exitCode=0
Sep 08 13:48:58.857642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.853032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" event={"ID":"2fa5c8a0-7d0e-4652-96b7-192e025a7a15","Type":"ContainerDied","Data":"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"}
Sep 08 13:48:58.857642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.853115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n" event={"ID":"2fa5c8a0-7d0e-4652-96b7-192e025a7a15","Type":"ContainerDied","Data":"deb9f723993d4a8348b585f0d393d3280a823dcb8bf7f4248bdf74904d2029ff"}
Sep 08 13:48:58.857642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.853408    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"
Sep 08 13:48:58.864708 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.864686    2577 scope.go:117] "RemoveContainer" containerID="cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"
Sep 08 13:48:58.865030 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:48:58.865006    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c\": container with ID starting with cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c not found: ID does not exist" containerID="cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"
Sep 08 13:48:58.865114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.865037    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c"} err="failed to get container status \"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c\": rpc error: code = NotFound desc = could not find container \"cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c\": container with ID starting with cb64a502ebed4a96dd89718aee2ab5d1b0b7ba103ccc5b79e1b0ab15e471644c not found: ID does not exist"
Sep 08 13:48:58.865114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.865054    2577 scope.go:117] "RemoveContainer" containerID="8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"
Sep 08 13:48:58.873396 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.873370    2577 scope.go:117] "RemoveContainer" containerID="8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"
Sep 08 13:48:58.873666 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:48:58.873647    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc\": container with ID starting with 8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc not found: ID does not exist" containerID="8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"
Sep 08 13:48:58.873718 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.873673    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc"} err="failed to get container status \"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc\": rpc error: code = NotFound desc = could not find container \"8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc\": container with ID starting with 8ae2485252863e76eb0cd3b79ad9a31c59b205cbc2e74ed16a6f7af1076f04bc not found: ID does not exist"
Sep 08 13:48:58.888191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.888161    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:48:58.893586 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.893563    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-2d1742-7d576c9db6-78d7n"]
Sep 08 13:48:58.907248 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.907216    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:48:58.914409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:58.914380    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-2d1742-585bd4fb85-r9ngj"]
Sep 08 13:48:59.213407 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:59.213370    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" path="/var/lib/kubelet/pods/2fa5c8a0-7d0e-4652-96b7-192e025a7a15/volumes"
Sep 08 13:48:59.213777 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:59.213763    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" path="/var/lib/kubelet/pods/ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a/volumes"
Sep 08 13:48:59.214105 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:48:59.214091    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" path="/var/lib/kubelet/pods/f4ad22d5-a6ba-460c-b779-ba2b39328fb8/volumes"
Sep 08 13:49:05.831281 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831246    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"]
Sep 08 13:49:05.831722 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831707    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" containerName="payload-pre-processing"
Sep 08 13:49:05.831767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831724    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" containerName="payload-pre-processing"
Sep 08 13:49:05.831767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831737    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" containerName="payload-processing"
Sep 08 13:49:05.831767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831742    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" containerName="payload-processing"
Sep 08 13:49:05.831767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831750    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" containerName="revoke-keys"
Sep 08 13:49:05.831767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831755    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" containerName="revoke-keys"
Sep 08 13:49:05.831914 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831818    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa5c8a0-7d0e-4652-96b7-192e025a7a15" containerName="payload-pre-processing"
Sep 08 13:49:05.831914 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831828    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ad22d5-a6ba-460c-b779-ba2b39328fb8" containerName="payload-processing"
Sep 08 13:49:05.831914 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.831837    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef244f3c-3fff-4e22-aa3b-dc9b8a01f96a" containerName="revoke-keys"
Sep 08 13:49:05.834970 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.834955    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:05.837162 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.837142    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:49:05.843727 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.843704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"]
Sep 08 13:49:05.892812 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.892784    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8154283-vwmrr\" (UID: \"2fee5717-a0a0-4d7e-9c93-e285f4d95c84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:05.993729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.993688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8154283-vwmrr\" (UID: \"2fee5717-a0a0-4d7e-9c93-e285f4d95c84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:05.994287 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:05.994266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f8154283-vwmrr\" (UID: \"2fee5717-a0a0-4d7e-9c93-e285f4d95c84\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:06.146699 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:06.146670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:06.416208 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:06.416185    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"]
Sep 08 13:49:06.418903 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:49:06.418868    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fee5717_a0a0_4d7e_9c93_e285f4d95c84.slice/crio-33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830 WatchSource:0}: Error finding container 33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830: Status 404 returned error can't find the container with id 33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830
Sep 08 13:49:06.898464 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:06.898432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr" event={"ID":"2fee5717-a0a0-4d7e-9c93-e285f4d95c84","Type":"ContainerStarted","Data":"72042cf11a6790a1ffac744ddaaa604dc2033f68b3a75857cc8fe55ebf0d04d1"}
Sep 08 13:49:06.898826 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:06.898486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr" event={"ID":"2fee5717-a0a0-4d7e-9c93-e285f4d95c84","Type":"ContainerStarted","Data":"33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830"}
Sep 08 13:49:06.917762 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:06.917713    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr" podStartSLOduration=1.917697261 podStartE2EDuration="1.917697261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:49:06.915430901 +0000 UTC m=+1230.231834474" watchObservedRunningTime="2026-09-08 13:49:06.917697261 +0000 UTC m=+1230.234100852"
Sep 08 13:49:07.905313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:07.905277    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fee5717-a0a0-4d7e-9c93-e285f4d95c84" containerID="72042cf11a6790a1ffac744ddaaa604dc2033f68b3a75857cc8fe55ebf0d04d1" exitCode=0
Sep 08 13:49:07.905703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:07.905367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr" event={"ID":"2fee5717-a0a0-4d7e-9c93-e285f4d95c84","Type":"ContainerDied","Data":"72042cf11a6790a1ffac744ddaaa604dc2033f68b3a75857cc8fe55ebf0d04d1"}
Sep 08 13:49:09.036448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.036427    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:09.115153 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.115121    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca\") pod \"2fee5717-a0a0-4d7e-9c93-e285f4d95c84\" (UID: \"2fee5717-a0a0-4d7e-9c93-e285f4d95c84\") "
Sep 08 13:49:09.115513 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.115491    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fee5717-a0a0-4d7e-9c93-e285f4d95c84" (UID: "2fee5717-a0a0-4d7e-9c93-e285f4d95c84"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:09.216821 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.216666    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fee5717-a0a0-4d7e-9c93-e285f4d95c84-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:09.916937 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.916907    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"
Sep 08 13:49:09.916937 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.916924    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr" event={"ID":"2fee5717-a0a0-4d7e-9c93-e285f4d95c84","Type":"ContainerDied","Data":"33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830"}
Sep 08 13:49:09.917195 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:09.916994    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33602672baae203aa53bd094b93736f590358778a1a5cf9dad869f1ccbbc6830"
Sep 08 13:49:16.156378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.156324    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:49:16.156791 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.156596    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" podUID="044a8c52-0528-41cd-bfab-d93b39be2adb" containerName="payload-pre-processing" containerID="cri-o://c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc" gracePeriod=30
Sep 08 13:49:16.275947 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.275919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"]
Sep 08 13:49:16.281917 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.281891    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a7ef4b-29814585-jhs29"]
Sep 08 13:49:16.314038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.314007    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:49:16.314392 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.314205    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" podUID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" containerName="payload-processing" containerID="cri-o://98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0" gracePeriod=30
Sep 08 13:49:16.321129 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.321093    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:49:16.321611 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.321559    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" podUID="6c41050c-d4b4-45b2-b627-223a794f78fc" containerName="payload-pre-processing" containerID="cri-o://4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5" gracePeriod=30
Sep 08 13:49:16.350594 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.350568    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"]
Sep 08 13:49:16.353162 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.353137    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f8154283-vwmrr"]
Sep 08 13:49:16.451599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.451576    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:49:16.486064 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.486017    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxx4j\" (UniqueName: \"kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j\") pod \"044a8c52-0528-41cd-bfab-d93b39be2adb\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") "
Sep 08 13:49:16.486244 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.486131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume\") pod \"044a8c52-0528-41cd-bfab-d93b39be2adb\" (UID: \"044a8c52-0528-41cd-bfab-d93b39be2adb\") "
Sep 08 13:49:16.486600 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.486571    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "044a8c52-0528-41cd-bfab-d93b39be2adb" (UID: "044a8c52-0528-41cd-bfab-d93b39be2adb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:16.488761 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.488735    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j" (OuterVolumeSpecName: "kube-api-access-kxx4j") pod "044a8c52-0528-41cd-bfab-d93b39be2adb" (UID: "044a8c52-0528-41cd-bfab-d93b39be2adb"). InnerVolumeSpecName "kube-api-access-kxx4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:16.587208 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.587186    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:49:16.587494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.587473    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxx4j\" (UniqueName: \"kubernetes.io/projected/044a8c52-0528-41cd-bfab-d93b39be2adb-kube-api-access-kxx4j\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.587541 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.587500    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/044a8c52-0528-41cd-bfab-d93b39be2adb-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.595635 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.595615    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:49:16.688717 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.688637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drw6b\" (UniqueName: \"kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b\") pod \"6c41050c-d4b4-45b2-b627-223a794f78fc\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") "
Sep 08 13:49:16.688867 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.688735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume\") pod \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") "
Sep 08 13:49:16.688867 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.688824    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45hmg\" (UniqueName: \"kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg\") pod \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\" (UID: \"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef\") "
Sep 08 13:49:16.688867 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.688861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume\") pod \"6c41050c-d4b4-45b2-b627-223a794f78fc\" (UID: \"6c41050c-d4b4-45b2-b627-223a794f78fc\") "
Sep 08 13:49:16.689096 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.689069    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" (UID: "6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:16.689276 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.689258    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.689276 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.689263    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c41050c-d4b4-45b2-b627-223a794f78fc" (UID: "6c41050c-d4b4-45b2-b627-223a794f78fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:16.690956 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.690936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b" (OuterVolumeSpecName: "kube-api-access-drw6b") pod "6c41050c-d4b4-45b2-b627-223a794f78fc" (UID: "6c41050c-d4b4-45b2-b627-223a794f78fc"). InnerVolumeSpecName "kube-api-access-drw6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:16.690956 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.690948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg" (OuterVolumeSpecName: "kube-api-access-45hmg") pod "6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" (UID: "6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef"). InnerVolumeSpecName "kube-api-access-45hmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:16.790555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.790519    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drw6b\" (UniqueName: \"kubernetes.io/projected/6c41050c-d4b4-45b2-b627-223a794f78fc-kube-api-access-drw6b\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.790555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.790545    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45hmg\" (UniqueName: \"kubernetes.io/projected/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef-kube-api-access-45hmg\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.790555 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.790554    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c41050c-d4b4-45b2-b627-223a794f78fc-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.954834 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.954740    2577 generic.go:356] "Generic (PLEG): container finished" podID="6c41050c-d4b4-45b2-b627-223a794f78fc" containerID="4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5" exitCode=0
Sep 08 13:49:16.954834 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.954790    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"
Sep 08 13:49:16.954834 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.954821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" event={"ID":"6c41050c-d4b4-45b2-b627-223a794f78fc","Type":"ContainerDied","Data":"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"}
Sep 08 13:49:16.955095 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.954876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph" event={"ID":"6c41050c-d4b4-45b2-b627-223a794f78fc","Type":"ContainerDied","Data":"4a48b090095eb70a3c9fc001352348e701d31cc4fc7fbea44239ccdd942029af"}
Sep 08 13:49:16.955095 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.954904    2577 scope.go:117] "RemoveContainer" containerID="4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"
Sep 08 13:49:16.956268 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.956247    2577 generic.go:356] "Generic (PLEG): container finished" podID="044a8c52-0528-41cd-bfab-d93b39be2adb" containerID="c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc" exitCode=0
Sep 08 13:49:16.956386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.956286    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"
Sep 08 13:49:16.956386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.956333    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" event={"ID":"044a8c52-0528-41cd-bfab-d93b39be2adb","Type":"ContainerDied","Data":"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"}
Sep 08 13:49:16.956471 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.956413    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg" event={"ID":"044a8c52-0528-41cd-bfab-d93b39be2adb","Type":"ContainerDied","Data":"8e5d9f7439d8b8bc13ed0c6a70bc12a94203afd6a39ad177db1b7f2bbe157bc5"}
Sep 08 13:49:16.958047 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.958024    2577 generic.go:356] "Generic (PLEG): container finished" podID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" containerID="98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0" exitCode=0
Sep 08 13:49:16.958183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.958052    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" event={"ID":"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef","Type":"ContainerDied","Data":"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"}
Sep 08 13:49:16.958183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.958070    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"
Sep 08 13:49:16.958183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.958101    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz" event={"ID":"6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef","Type":"ContainerDied","Data":"f47ab8812de85799d08532d08ca8bb7f20f917f753e3c67ac847a3663bb2b326"}
Sep 08 13:49:16.964780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.964749    2577 scope.go:117] "RemoveContainer" containerID="4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"
Sep 08 13:49:16.965060 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:49:16.965036    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5\": container with ID starting with 4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5 not found: ID does not exist" containerID="4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"
Sep 08 13:49:16.965146 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.965069    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5"} err="failed to get container status \"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5\": rpc error: code = NotFound desc = could not find container \"4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5\": container with ID starting with 4b662a137380c596c562cd3d6aa9ec90c3315f5d77641b486bba48ee303471f5 not found: ID does not exist"
Sep 08 13:49:16.965146 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.965094    2577 scope.go:117] "RemoveContainer" containerID="c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"
Sep 08 13:49:16.973697 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.973681    2577 scope.go:117] "RemoveContainer" containerID="c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"
Sep 08 13:49:16.973968 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:49:16.973950    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc\": container with ID starting with c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc not found: ID does not exist" containerID="c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"
Sep 08 13:49:16.974038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.973976    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc"} err="failed to get container status \"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc\": rpc error: code = NotFound desc = could not find container \"c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc\": container with ID starting with c927d84857e1180872cbdba43fa3f6a19e98c553fb27a135f00d52b2fc0c7fbc not found: ID does not exist"
Sep 08 13:49:16.974038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.973990    2577 scope.go:117] "RemoveContainer" containerID="98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"
Sep 08 13:49:16.982909 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.982895    2577 scope.go:117] "RemoveContainer" containerID="98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"
Sep 08 13:49:16.983150 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:49:16.983129    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0\": container with ID starting with 98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0 not found: ID does not exist" containerID="98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"
Sep 08 13:49:16.983213 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:16.983158    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0"} err="failed to get container status \"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0\": rpc error: code = NotFound desc = could not find container \"98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0\": container with ID starting with 98948a853a24660795fa02b6cf54a788ddc22a389d0f335547dda638ecf7ede0 not found: ID does not exist"
Sep 08 13:49:17.004308 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.004283    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:49:17.015427 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.015405    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a7ef4b-79d5c59b57-g82qz"]
Sep 08 13:49:17.034582 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.034557    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:49:17.040905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.040885    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f8154283-7794fd6c74-m6stg"]
Sep 08 13:49:17.050461 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.050436    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:49:17.056157 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.056136    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a7ef4b-7dbfc475f8-7kxph"]
Sep 08 13:49:17.214124 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.214042    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="044a8c52-0528-41cd-bfab-d93b39be2adb" path="/var/lib/kubelet/pods/044a8c52-0528-41cd-bfab-d93b39be2adb/volumes"
Sep 08 13:49:17.214463 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.214328    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="148dfbb4-98cd-43ef-820b-f8b2e1ea2e75" path="/var/lib/kubelet/pods/148dfbb4-98cd-43ef-820b-f8b2e1ea2e75/volumes"
Sep 08 13:49:17.214607 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.214593    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fee5717-a0a0-4d7e-9c93-e285f4d95c84" path="/var/lib/kubelet/pods/2fee5717-a0a0-4d7e-9c93-e285f4d95c84/volumes"
Sep 08 13:49:17.214885 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.214875    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c41050c-d4b4-45b2-b627-223a794f78fc" path="/var/lib/kubelet/pods/6c41050c-d4b4-45b2-b627-223a794f78fc/volumes"
Sep 08 13:49:17.216120 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:17.216087    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" path="/var/lib/kubelet/pods/6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef/volumes"
Sep 08 13:49:38.494372 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494318    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:49:38.494916 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494897    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c41050c-d4b4-45b2-b627-223a794f78fc" containerName="payload-pre-processing"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494919    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c41050c-d4b4-45b2-b627-223a794f78fc" containerName="payload-pre-processing"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494934    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" containerName="payload-processing"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494943    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" containerName="payload-processing"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494972    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fee5717-a0a0-4d7e-9c93-e285f4d95c84" containerName="revoke-keys"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494980    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fee5717-a0a0-4d7e-9c93-e285f4d95c84" containerName="revoke-keys"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.494995    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="044a8c52-0528-41cd-bfab-d93b39be2adb" containerName="payload-pre-processing"
Sep 08 13:49:38.495004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.495002    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="044a8c52-0528-41cd-bfab-d93b39be2adb" containerName="payload-pre-processing"
Sep 08 13:49:38.495369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.495104    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fbe5639-89e2-4ae1-8261-2e8b4cfb91ef" containerName="payload-processing"
Sep 08 13:49:38.495369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.495119    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c41050c-d4b4-45b2-b627-223a794f78fc" containerName="payload-pre-processing"
Sep 08 13:49:38.495369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.495134    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fee5717-a0a0-4d7e-9c93-e285f4d95c84" containerName="revoke-keys"
Sep 08 13:49:38.495369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.495145    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="044a8c52-0528-41cd-bfab-d93b39be2adb" containerName="payload-pre-processing"
Sep 08 13:49:38.499204 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.499185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.501501 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.501481    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-3c9fc8-openshift-default-dockercfg-hlf67\""
Sep 08 13:49:38.513171 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.513150    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:49:38.598141 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598273    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598307 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2s5t\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.598503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.598442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.699822 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.699782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.699848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.699895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.699925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700015 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.699973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2s5t\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700467    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.700629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.700609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.702540 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.702512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.702649 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.702601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.707915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.707890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2s5t\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.708426 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.708409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token\") pod \"e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") " pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:38.809196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:38.809101    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:39.085330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:39.085302    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:49:39.087230 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:49:39.087207    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac385d3_571f_4d16_86c4_9562c6a73223.slice/crio-ae5117d5484f3bcdaaa1be1754291180053ff685024df9fd86f0c96ecfbdd8ce WatchSource:0}: Error finding container ae5117d5484f3bcdaaa1be1754291180053ff685024df9fd86f0c96ecfbdd8ce: Status 404 returned error can't find the container with id ae5117d5484f3bcdaaa1be1754291180053ff685024df9fd86f0c96ecfbdd8ce
Sep 08 13:49:39.089736 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:39.089705    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:39.089808 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:39.089768    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:39.089847 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:39.089822    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:40.078832 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:40.078794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerStarted","Data":"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"}
Sep 08 13:49:40.079203 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:40.078846    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerStarted","Data":"ae5117d5484f3bcdaaa1be1754291180053ff685024df9fd86f0c96ecfbdd8ce"}
Sep 08 13:49:40.099287 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:40.099235    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podStartSLOduration=2.099220308 podStartE2EDuration="2.099220308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:49:40.097953845 +0000 UTC m=+1263.414357399" watchObservedRunningTime="2026-09-08 13:49:40.099220308 +0000 UTC m=+1263.415623860"
Sep 08 13:49:40.809327 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:40.809288    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:40.814300 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:40.814275    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:41.083668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:41.083575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:41.084587 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:41.084569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:44.140013 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.139981    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:49:44.145203 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.145177    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.147322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.147288    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-3c9fc8-dockercfg-c7jvf\""
Sep 08 13:49:44.147466 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.147333    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-3c9fc8\""
Sep 08 13:49:44.157920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.157896    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:49:44.253889 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.253834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.254083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.253978    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkfpv\" (UniqueName: \"kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.356467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.355503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.356664 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.356428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.356664 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.356621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkfpv\" (UniqueName: \"kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.365323 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.365291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkfpv\" (UniqueName: \"kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv\") pod \"payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.457871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.457781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:44.792768 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:44.792680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:49:44.804259 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:49:44.804232    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050ee39f_4fd4_498a_8215_dc8898fe0bc8.slice/crio-b3945ba91b5f2fa94bb690bc4396331ee43704b4f57bfb76363cc4404d3ced91 WatchSource:0}: Error finding container b3945ba91b5f2fa94bb690bc4396331ee43704b4f57bfb76363cc4404d3ced91: Status 404 returned error can't find the container with id b3945ba91b5f2fa94bb690bc4396331ee43704b4f57bfb76363cc4404d3ced91
Sep 08 13:49:45.109174 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:45.108937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" event={"ID":"050ee39f-4fd4-498a-8215-dc8898fe0bc8","Type":"ContainerStarted","Data":"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"}
Sep 08 13:49:45.109174 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:45.109020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" event={"ID":"050ee39f-4fd4-498a-8215-dc8898fe0bc8","Type":"ContainerStarted","Data":"b3945ba91b5f2fa94bb690bc4396331ee43704b4f57bfb76363cc4404d3ced91"}
Sep 08 13:49:45.109174 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:45.109050    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:45.127218 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:45.127168    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" podStartSLOduration=1.127147792 podStartE2EDuration="1.127147792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:49:45.126037765 +0000 UTC m=+1268.442441335" watchObservedRunningTime="2026-09-08 13:49:45.127147792 +0000 UTC m=+1268.443551343"
Sep 08 13:49:56.117773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:56.117735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:49:57.126511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:57.126477    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 13:49:57.126843 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:57.126540    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 13:49:58.176916 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.176835    2577 generic.go:356] "Generic (PLEG): container finished" podID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerID="cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275" exitCode=0
Sep 08 13:49:58.176916 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.176905    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerDied","Data":"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"}
Sep 08 13:49:58.177373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.177290    2577 scope.go:117] "RemoveContainer" containerID="cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"
Sep 08 13:49:58.178494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.178465    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:58.178585 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.178533    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:58.178585 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:58.178566    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:49:59.193912 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.192966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerStarted","Data":"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"}
Sep 08 13:49:59.809683 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.809643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:49:59.809895 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.809873    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 08 13:49:59.809951 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.809932    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 08 13:49:59.833388 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.833309    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 13:49:59.837109 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.837088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:49:59.839165 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.839143    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-905fec\""
Sep 08 13:49:59.839284 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.839216    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-905fec-dockercfg-48mqn\""
Sep 08 13:49:59.853312 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.853289    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 13:49:59.905875 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.905837    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hxsj\" (UniqueName: \"kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:49:59.906249 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:49:59.906223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.007075 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.007024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.007280 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.007128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxsj\" (UniqueName: \"kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.008370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.008326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.014912 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.014892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxsj\" (UniqueName: \"kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj\") pod \"payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.148028 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.147982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:00.443613 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.443584    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 13:50:00.446023 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:50:00.445992    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955d4192_7edb_4f9d_8842_7c3d09bbef11.slice/crio-27637848ee2efc1d0d34a7c17ac779d13d2b681400e5aebbfc6d159a1de9a7af WatchSource:0}: Error finding container 27637848ee2efc1d0d34a7c17ac779d13d2b681400e5aebbfc6d159a1de9a7af: Status 404 returned error can't find the container with id 27637848ee2efc1d0d34a7c17ac779d13d2b681400e5aebbfc6d159a1de9a7af
Sep 08 13:50:00.810284 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.810199    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 08 13:50:00.810284 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:00.810254    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 08 13:50:01.215766 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.215729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" event={"ID":"955d4192-7edb-4f9d-8842-7c3d09bbef11","Type":"ContainerStarted","Data":"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"}
Sep 08 13:50:01.216025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.215804    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:01.216025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.215826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" event={"ID":"955d4192-7edb-4f9d-8842-7c3d09bbef11","Type":"ContainerStarted","Data":"27637848ee2efc1d0d34a7c17ac779d13d2b681400e5aebbfc6d159a1de9a7af"}
Sep 08 13:50:01.229086 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.229011    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" podStartSLOduration=2.2289950689999998 podStartE2EDuration="2.228995069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:01.228010168 +0000 UTC m=+1284.544413758" watchObservedRunningTime="2026-09-08 13:50:01.228995069 +0000 UTC m=+1284.545398621"
Sep 08 13:50:01.810019 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.809981    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 08 13:50:01.810465 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:01.810046    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 08 13:50:02.809918 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:02.809876    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 08 13:50:02.810121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:02.809939    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 08 13:50:03.810153 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:03.810112    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 08 13:50:03.810569 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:03.810174    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 08 13:50:04.813700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:04.813668    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:50:04.814160 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:04.814045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:50:04.814784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:04.814724    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:50:12.219773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.219740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 13:50:12.694781 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.694751    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:50:12.698732 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.698712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.700687 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.700666    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-d9970e\""
Sep 08 13:50:12.708488 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.708469    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:50:12.836060 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.836019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.836389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.836129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxtdx\" (UniqueName: \"kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.937238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.937191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxtdx\" (UniqueName: \"kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.937432 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.937301    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.938558 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.938531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:12.949210 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:12.949145    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxtdx\" (UniqueName: \"kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx\") pod \"payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:13.009653 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:13.009626    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:13.285564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:13.285539    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:50:13.287399 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:50:13.287368    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod803c5489_a1df_41e6_ab48_399ccab42ddb.slice/crio-91c3cbc7e64dc2ba857c359dcae3e5a88faf8842f6d2f20d9a704d5cbbbc5f4e WatchSource:0}: Error finding container 91c3cbc7e64dc2ba857c359dcae3e5a88faf8842f6d2f20d9a704d5cbbbc5f4e: Status 404 returned error can't find the container with id 91c3cbc7e64dc2ba857c359dcae3e5a88faf8842f6d2f20d9a704d5cbbbc5f4e
Sep 08 13:50:14.282134 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:14.282098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" event={"ID":"803c5489-a1df-41e6-ab48-399ccab42ddb","Type":"ContainerStarted","Data":"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"}
Sep 08 13:50:14.282328 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:14.282157    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" event={"ID":"803c5489-a1df-41e6-ab48-399ccab42ddb","Type":"ContainerStarted","Data":"91c3cbc7e64dc2ba857c359dcae3e5a88faf8842f6d2f20d9a704d5cbbbc5f4e"}
Sep 08 13:50:14.282328 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:14.282198    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:14.301221 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:14.301167    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" podStartSLOduration=2.301148779 podStartE2EDuration="2.301148779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:14.300951394 +0000 UTC m=+1297.617354978" watchObservedRunningTime="2026-09-08 13:50:14.301148779 +0000 UTC m=+1297.617552331"
Sep 08 13:50:25.290468 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:25.290439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:50:48.914610 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.914580    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:50:48.917889 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.917875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.920042 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.920022    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2ee58a5f-openshift-default-dockercfg-nwxgx\""
Sep 08 13:50:48.932325 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.932304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:50:48.979077 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979050    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcf28\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:48.979511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:48.979421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080094 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080283    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080775 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.080969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.080951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.081222 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.081193    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcf28\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.081774 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.081753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.082083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.081431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.082172 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.081200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.088229 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.088065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.088692 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.088644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.090816 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.090794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.090932 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.090809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcf28\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28\") pod \"e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") " pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.228592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.228498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:49.503546 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.503511    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:50:49.505617 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:50:49.505588    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89825189_537f_4797_8ef4_6080e03a932f.slice/crio-dc0417b03ff89569dd8b25ec51be97155be38cbc96c8f1a38bfb49cabc31d483 WatchSource:0}: Error finding container dc0417b03ff89569dd8b25ec51be97155be38cbc96c8f1a38bfb49cabc31d483: Status 404 returned error can't find the container with id dc0417b03ff89569dd8b25ec51be97155be38cbc96c8f1a38bfb49cabc31d483
Sep 08 13:50:49.509378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.508010    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:50:49.509378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.508302    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:50:49.509378 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:49.508417    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:50:50.474935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:50.474902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerStarted","Data":"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"}
Sep 08 13:50:50.475433 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:50.474997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerStarted","Data":"dc0417b03ff89569dd8b25ec51be97155be38cbc96c8f1a38bfb49cabc31d483"}
Sep 08 13:50:50.497917 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:50.497866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podStartSLOduration=2.497849978 podStartE2EDuration="2.497849978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:50.495827625 +0000 UTC m=+1333.812231202" watchObservedRunningTime="2026-09-08 13:50:50.497849978 +0000 UTC m=+1333.814253530"
Sep 08 13:50:51.229000 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:51.228971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:51.234673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:51.234652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:51.479889 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:51.479802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:51.480769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:51.480743    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:50:54.435231 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.435195    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:50:54.439011 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.438983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.442402 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.442378    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2ee58a5f-dockercfg-lp598\""
Sep 08 13:50:54.442535 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.442381    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2ee58a5f\""
Sep 08 13:50:54.449640 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.449620    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:50:54.475878 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.475856    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:50:54.479120 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.479107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.502617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.502595    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:50:54.539789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.539736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmqnl\" (UniqueName: \"kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.539976 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.539812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.540049 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.539966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc6fs\" (UniqueName: \"kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.540049 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.540032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.641038 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.640998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6fs\" (UniqueName: \"kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.641228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.641062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.641228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.641140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmqnl\" (UniqueName: \"kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.641228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.641187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.641831 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.641805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.641938 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.641914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.649781 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.649751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmqnl\" (UniqueName: \"kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl\") pod \"payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") " pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:54.649870 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.649821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6fs\" (UniqueName: \"kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs\") pod \"payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.751003 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.750919    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:54.792776 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:54.789542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:55.056141 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.056112    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:50:55.057518 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:50:55.057484    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95d237b_1ee9_4509_bba3_800d7f59688f.slice/crio-238844e76b64febe7f52eb9dc2807994e121b0bf855940eea998841a643f3daf WatchSource:0}: Error finding container 238844e76b64febe7f52eb9dc2807994e121b0bf855940eea998841a643f3daf: Status 404 returned error can't find the container with id 238844e76b64febe7f52eb9dc2807994e121b0bf855940eea998841a643f3daf
Sep 08 13:50:55.090057 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.090031    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:50:55.095544 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:50:55.095405    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d39f808_035b_49ed_b76a_718fc380bb39.slice/crio-cc456a9b04cc18a50bfe113bca050e69054a7d634885a2efbe0b3b8a062e6d23 WatchSource:0}: Error finding container cc456a9b04cc18a50bfe113bca050e69054a7d634885a2efbe0b3b8a062e6d23: Status 404 returned error can't find the container with id cc456a9b04cc18a50bfe113bca050e69054a7d634885a2efbe0b3b8a062e6d23
Sep 08 13:50:55.505416 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.505374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" event={"ID":"b95d237b-1ee9-4509-bba3-800d7f59688f","Type":"ContainerStarted","Data":"af5838b11c10b4c809ebaeb8e4996b8b983097ff5b0db0c04a9a2ee84b2ec76d"}
Sep 08 13:50:55.505890 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.505436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" event={"ID":"b95d237b-1ee9-4509-bba3-800d7f59688f","Type":"ContainerStarted","Data":"238844e76b64febe7f52eb9dc2807994e121b0bf855940eea998841a643f3daf"}
Sep 08 13:50:55.505890 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.505572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:50:55.507486 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.507458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" event={"ID":"7d39f808-035b-49ed-b76a-718fc380bb39","Type":"ContainerStarted","Data":"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"}
Sep 08 13:50:55.507598 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.507500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" event={"ID":"7d39f808-035b-49ed-b76a-718fc380bb39","Type":"ContainerStarted","Data":"cc456a9b04cc18a50bfe113bca050e69054a7d634885a2efbe0b3b8a062e6d23"}
Sep 08 13:50:55.507598 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.507537    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:50:55.524722 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.524663    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" podStartSLOduration=1.5246434629999999 podStartE2EDuration="1.524643463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:55.521088138 +0000 UTC m=+1338.837491719" watchObservedRunningTime="2026-09-08 13:50:55.524643463 +0000 UTC m=+1338.841047017"
Sep 08 13:50:55.539796 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:50:55.539751    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" podStartSLOduration=1.539736346 podStartE2EDuration="1.539736346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:55.537418028 +0000 UTC m=+1338.853821603" watchObservedRunningTime="2026-09-08 13:50:55.539736346 +0000 UTC m=+1338.856139952"
Sep 08 13:51:06.514875 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:06.514802    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:51:06.515228 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:06.514878    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:51:11.597507 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.597420    2577 generic.go:356] "Generic (PLEG): container finished" podID="89825189-537f-4797-8ef4-6080e03a932f" containerID="931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b" exitCode=0
Sep 08 13:51:11.597507 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.597475    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerDied","Data":"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"}
Sep 08 13:51:11.597916 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.597851    2577 scope.go:117] "RemoveContainer" containerID="931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"
Sep 08 13:51:11.599019 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.598979    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:51:11.599194 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.599058    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:51:11.599194 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:11.599099    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:51:12.607967 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:12.607928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerStarted","Data":"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"}
Sep 08 13:51:13.229227 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:13.229199    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:51:13.229451 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:13.229421    2577 patch_prober.go:28] interesting pod/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 08 13:51:13.229529 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:13.229493    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 08 13:51:14.229903 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:14.229867    2577 patch_prober.go:28] interesting pod/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 08 13:51:14.230300 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:14.229936    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 08 13:51:15.229124 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:15.229091    2577 patch_prober.go:28] interesting pod/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 08 13:51:15.229294 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:15.229142    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 08 13:51:16.229115 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:16.229081    2577 patch_prober.go:28] interesting pod/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 08 13:51:16.229499 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:16.229132    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 08 13:51:17.228990 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:17.228951    2577 patch_prober.go:28] interesting pod/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 08 13:51:17.229154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:17.229009    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 08 13:51:18.233337 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:18.233306    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:51:18.233741 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:18.233609    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:51:18.234233 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:18.234216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:51:37.433722 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:51:37.433693    2577 scope.go:117] "RemoveContainer" containerID="84440671db2104666dc5c1319a58217341d9e6b892b06043925573b5ce8fa380"
Sep 08 13:52:13.249104 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.249068    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"]
Sep 08 13:52:13.257767 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.257733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:13.260457 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.260429    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:52:13.261297 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.261271    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"]
Sep 08 13:52:13.328216 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.328170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9\" (UID: \"0ea2a196-75c2-4727-b327-fa43bbc23ec9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:13.429377 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.429334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9\" (UID: \"0ea2a196-75c2-4727-b327-fa43bbc23ec9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:13.429957 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.429938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9\" (UID: \"0ea2a196-75c2-4727-b327-fa43bbc23ec9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:13.571617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.571510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:13.843570 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.843540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"]
Sep 08 13:52:13.845435 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:52:13.845397    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea2a196_75c2_4727_b327_fa43bbc23ec9.slice/crio-03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8 WatchSource:0}: Error finding container 03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8: Status 404 returned error can't find the container with id 03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8
Sep 08 13:52:13.951510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:13.951472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9" event={"ID":"0ea2a196-75c2-4727-b327-fa43bbc23ec9","Type":"ContainerStarted","Data":"03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8"}
Sep 08 13:52:14.958379 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:14.958268    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ea2a196-75c2-4727-b327-fa43bbc23ec9" containerID="63365c8eeb7dc2774f109eb5f92f074f6c10e63847a002ae76884062b5016d5e" exitCode=0
Sep 08 13:52:14.958725 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:14.958368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9" event={"ID":"0ea2a196-75c2-4727-b327-fa43bbc23ec9","Type":"ContainerDied","Data":"63365c8eeb7dc2774f109eb5f92f074f6c10e63847a002ae76884062b5016d5e"}
Sep 08 13:52:16.112269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.112244    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:16.259531 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.259423    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca\") pod \"0ea2a196-75c2-4727-b327-fa43bbc23ec9\" (UID: \"0ea2a196-75c2-4727-b327-fa43bbc23ec9\") "
Sep 08 13:52:16.259789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.259765    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ea2a196-75c2-4727-b327-fa43bbc23ec9" (UID: "0ea2a196-75c2-4727-b327-fa43bbc23ec9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:52:16.360811 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.360767    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea2a196-75c2-4727-b327-fa43bbc23ec9-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:52:16.975055 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.975030    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"
Sep 08 13:52:16.975227 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.975063    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9" event={"ID":"0ea2a196-75c2-4727-b327-fa43bbc23ec9","Type":"ContainerDied","Data":"03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8"}
Sep 08 13:52:16.975227 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:16.975114    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03dae52b21e949e4cc21db37f0fbc12e73bb8e9f6dac734be00766918c8495e8"
Sep 08 13:52:28.771450 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:28.771365    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:52:28.772005 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:28.771614    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" podUID="7d39f808-035b-49ed-b76a-718fc380bb39" containerName="payload-processing" containerID="cri-o://530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3" gracePeriod=30
Sep 08 13:52:28.780402 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:28.780194    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:52:28.780494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:28.780446    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" podUID="b95d237b-1ee9-4509-bba3-800d7f59688f" containerName="payload-pre-processing" containerID="cri-o://af5838b11c10b4c809ebaeb8e4996b8b983097ff5b0db0c04a9a2ee84b2ec76d" gracePeriod=30
Sep 08 13:52:29.004481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.004416    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"]
Sep 08 13:52:29.006604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.006575    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ee58a5f-gvpv9"]
Sep 08 13:52:29.039175 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.039108    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:52:29.048624 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.048598    2577 generic.go:356] "Generic (PLEG): container finished" podID="7d39f808-035b-49ed-b76a-718fc380bb39" containerID="530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3" exitCode=0
Sep 08 13:52:29.048797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.048657    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"
Sep 08 13:52:29.048797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.048668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" event={"ID":"7d39f808-035b-49ed-b76a-718fc380bb39","Type":"ContainerDied","Data":"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"}
Sep 08 13:52:29.048797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.048755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl" event={"ID":"7d39f808-035b-49ed-b76a-718fc380bb39","Type":"ContainerDied","Data":"cc456a9b04cc18a50bfe113bca050e69054a7d634885a2efbe0b3b8a062e6d23"}
Sep 08 13:52:29.048797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.048783    2577 scope.go:117] "RemoveContainer" containerID="530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"
Sep 08 13:52:29.050273 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.050251    2577 generic.go:356] "Generic (PLEG): container finished" podID="b95d237b-1ee9-4509-bba3-800d7f59688f" containerID="af5838b11c10b4c809ebaeb8e4996b8b983097ff5b0db0c04a9a2ee84b2ec76d" exitCode=0
Sep 08 13:52:29.050422 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.050402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" event={"ID":"b95d237b-1ee9-4509-bba3-800d7f59688f","Type":"ContainerDied","Data":"af5838b11c10b4c809ebaeb8e4996b8b983097ff5b0db0c04a9a2ee84b2ec76d"}
Sep 08 13:52:29.059516 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.059492    2577 scope.go:117] "RemoveContainer" containerID="530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"
Sep 08 13:52:29.059890 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:52:29.059863    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3\": container with ID starting with 530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3 not found: ID does not exist" containerID="530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"
Sep 08 13:52:29.060011 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.059904    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3"} err="failed to get container status \"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3\": rpc error: code = NotFound desc = could not find container \"530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3\": container with ID starting with 530a85762fa4244319d73276e05b5c633120b1136803bb7778aed873d312e4e3 not found: ID does not exist"
Sep 08 13:52:29.081798 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.081771    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:52:29.091383 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091338    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc6fs\" (UniqueName: \"kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs\") pod \"b95d237b-1ee9-4509-bba3-800d7f59688f\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") "
Sep 08 13:52:29.091503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmqnl\" (UniqueName: \"kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl\") pod \"7d39f808-035b-49ed-b76a-718fc380bb39\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") "
Sep 08 13:52:29.091503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume\") pod \"b95d237b-1ee9-4509-bba3-800d7f59688f\" (UID: \"b95d237b-1ee9-4509-bba3-800d7f59688f\") "
Sep 08 13:52:29.091623 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091514    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume\") pod \"7d39f808-035b-49ed-b76a-718fc380bb39\" (UID: \"7d39f808-035b-49ed-b76a-718fc380bb39\") "
Sep 08 13:52:29.091887 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091845    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b95d237b-1ee9-4509-bba3-800d7f59688f" (UID: "b95d237b-1ee9-4509-bba3-800d7f59688f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:52:29.091974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.091896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d39f808-035b-49ed-b76a-718fc380bb39" (UID: "7d39f808-035b-49ed-b76a-718fc380bb39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:52:29.093575 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.093553    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs" (OuterVolumeSpecName: "kube-api-access-vc6fs") pod "b95d237b-1ee9-4509-bba3-800d7f59688f" (UID: "b95d237b-1ee9-4509-bba3-800d7f59688f"). InnerVolumeSpecName "kube-api-access-vc6fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:52:29.093679 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.093652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl" (OuterVolumeSpecName: "kube-api-access-tmqnl") pod "7d39f808-035b-49ed-b76a-718fc380bb39" (UID: "7d39f808-035b-49ed-b76a-718fc380bb39"). InnerVolumeSpecName "kube-api-access-tmqnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:52:29.192474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.192432    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d39f808-035b-49ed-b76a-718fc380bb39-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.192474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.192466    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc6fs\" (UniqueName: \"kubernetes.io/projected/b95d237b-1ee9-4509-bba3-800d7f59688f-kube-api-access-vc6fs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.192474 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.192478    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmqnl\" (UniqueName: \"kubernetes.io/projected/7d39f808-035b-49ed-b76a-718fc380bb39-kube-api-access-tmqnl\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.192713 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.192489    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b95d237b-1ee9-4509-bba3-800d7f59688f-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.213920 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.213887    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea2a196-75c2-4727-b327-fa43bbc23ec9" path="/var/lib/kubelet/pods/0ea2a196-75c2-4727-b327-fa43bbc23ec9/volumes"
Sep 08 13:52:29.364604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.364532    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:52:29.370816 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:29.370789    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ee58a5f-7d67f9d957-xflnl"]
Sep 08 13:52:30.057977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:30.057898    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"
Sep 08 13:52:30.057977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:30.057929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws" event={"ID":"b95d237b-1ee9-4509-bba3-800d7f59688f","Type":"ContainerDied","Data":"238844e76b64febe7f52eb9dc2807994e121b0bf855940eea998841a643f3daf"}
Sep 08 13:52:30.058467 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:30.057980    2577 scope.go:117] "RemoveContainer" containerID="af5838b11c10b4c809ebaeb8e4996b8b983097ff5b0db0c04a9a2ee84b2ec76d"
Sep 08 13:52:30.075238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:30.075213    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:52:30.081282 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:30.081262    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ee58a5f-7654fccfb6-4kzws"]
Sep 08 13:52:31.213835 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:31.213791    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d39f808-035b-49ed-b76a-718fc380bb39" path="/var/lib/kubelet/pods/7d39f808-035b-49ed-b76a-718fc380bb39/volumes"
Sep 08 13:52:31.214370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:31.214330    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b95d237b-1ee9-4509-bba3-800d7f59688f" path="/var/lib/kubelet/pods/b95d237b-1ee9-4509-bba3-800d7f59688f/volumes"
Sep 08 13:52:37.454202 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:52:37.454169    2577 scope.go:117] "RemoveContainer" containerID="8bea503dcab01098464c1faf82ae697059df6f15e9bcc9825fbad0c495424730"
Sep 08 13:53:20.824556 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:20.824520    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:53:20.825190 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:20.824765    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy" containerID="cri-o://77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a" gracePeriod=30
Sep 08 13:53:26.078508 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.078478    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:53:26.151198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151159    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151224    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcf28\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151267    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151299    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151333    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151510 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151495    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151536    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data" (OuterVolumeSpecName: "istio-data") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:53:26.151769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151545    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:53:26.151769 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151729    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:53:26.151935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151764    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:53:26.151935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151811    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.151935 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.151842    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token\") pod \"89825189-537f-4797-8ef4-6080e03a932f\" (UID: \"89825189-537f-4797-8ef4-6080e03a932f\") "
Sep 08 13:53:26.152261 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.152245    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.152321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.152266    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.152321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.152269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:53:26.152321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.152282    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89825189-537f-4797-8ef4-6080e03a932f-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.152321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.152296    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.153720 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.153686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:53:26.153720 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.153698    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28" (OuterVolumeSpecName: "kube-api-access-xcf28") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "kube-api-access-xcf28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:53:26.153895 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.153752    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 13:53:26.153952 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.153903    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token" (OuterVolumeSpecName: "istio-token") pod "89825189-537f-4797-8ef4-6080e03a932f" (UID: "89825189-537f-4797-8ef4-6080e03a932f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:53:26.253642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.253603    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.253642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.253634    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.253642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.253643    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcf28\" (UniqueName: \"kubernetes.io/projected/89825189-537f-4797-8ef4-6080e03a932f-kube-api-access-xcf28\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.253642 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.253653    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89825189-537f-4797-8ef4-6080e03a932f-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.253988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.253661    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89825189-537f-4797-8ef4-6080e03a932f-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:53:26.360590 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.360556    2577 generic.go:356] "Generic (PLEG): container finished" podID="89825189-537f-4797-8ef4-6080e03a932f" containerID="77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a" exitCode=0
Sep 08 13:53:26.360793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.360600    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"
Sep 08 13:53:26.360793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.360606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerDied","Data":"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"}
Sep 08 13:53:26.360793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.360660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6" event={"ID":"89825189-537f-4797-8ef4-6080e03a932f","Type":"ContainerDied","Data":"dc0417b03ff89569dd8b25ec51be97155be38cbc96c8f1a38bfb49cabc31d483"}
Sep 08 13:53:26.360793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.360690    2577 scope.go:117] "RemoveContainer" containerID="77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"
Sep 08 13:53:26.372731 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.372713    2577 scope.go:117] "RemoveContainer" containerID="931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"
Sep 08 13:53:26.381247 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.381228    2577 scope.go:117] "RemoveContainer" containerID="77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"
Sep 08 13:53:26.381621 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:53:26.381600    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a\": container with ID starting with 77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a not found: ID does not exist" containerID="77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"
Sep 08 13:53:26.381694 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.381630    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a"} err="failed to get container status \"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a\": rpc error: code = NotFound desc = could not find container \"77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a\": container with ID starting with 77a49bf9641856e3e739ea7f752a2eda3a3e056ace3ec5033497b1ff0991cf1a not found: ID does not exist"
Sep 08 13:53:26.381694 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.381648    2577 scope.go:117] "RemoveContainer" containerID="931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"
Sep 08 13:53:26.381909 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:53:26.381889    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b\": container with ID starting with 931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b not found: ID does not exist" containerID="931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"
Sep 08 13:53:26.381963 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.381919    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b"} err="failed to get container status \"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b\": rpc error: code = NotFound desc = could not find container \"931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b\": container with ID starting with 931fa312b4c2fd7fcced88b20dcb94d682296e2bd5a57edb3e3809474048dc8b not found: ID does not exist"
Sep 08 13:53:26.387966 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.387943    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:53:26.389919 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:26.389902    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2ee58a5f-openshift-default-57fb98dd56-djld6"]
Sep 08 13:53:27.214429 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:27.214395    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89825189-537f-4797-8ef4-6080e03a932f" path="/var/lib/kubelet/pods/89825189-537f-4797-8ef4-6080e03a932f/volumes"
Sep 08 13:53:37.243106 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:37.243067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:53:37.243580 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:53:37.243322    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:54:15.755604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755503    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755954    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d39f808-035b-49ed-b76a-718fc380bb39" containerName="payload-processing"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755966    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d39f808-035b-49ed-b76a-718fc380bb39" containerName="payload-processing"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755974    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755979    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755993    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b95d237b-1ee9-4509-bba3-800d7f59688f" containerName="payload-pre-processing"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.755999    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b95d237b-1ee9-4509-bba3-800d7f59688f" containerName="payload-pre-processing"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756005    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea2a196-75c2-4727-b327-fa43bbc23ec9" containerName="revoke-keys"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756011    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea2a196-75c2-4727-b327-fa43bbc23ec9" containerName="revoke-keys"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756025    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.756074 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756031    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.756448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756108    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d39f808-035b-49ed-b76a-718fc380bb39" containerName="payload-processing"
Sep 08 13:54:15.756448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756118    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea2a196-75c2-4727-b327-fa43bbc23ec9" containerName="revoke-keys"
Sep 08 13:54:15.756448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756124    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.756448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756129    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b95d237b-1ee9-4509-bba3-800d7f59688f" containerName="payload-pre-processing"
Sep 08 13:54:15.756448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.756136    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="89825189-537f-4797-8ef4-6080e03a932f" containerName="istio-proxy"
Sep 08 13:54:15.759617 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.759595    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.761810 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.761788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-569975-openshift-default-dockercfg-q62lq\""
Sep 08 13:54:15.772774 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.772745    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:54:15.846304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846313    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846679 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp6pn\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846679 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846679 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.846679 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.846673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947506 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947465    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.947734 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948040 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.947885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp6pn\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948449 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.948283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948449 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.948325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948449 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.948390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948652 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.948484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.948866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.948849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.950393 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.950373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.950686 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.950665    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.956219 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.956196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:15.957138 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:15.957115    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp6pn\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn\") pod \"e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") " pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:16.070910 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.070816    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:16.365239 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.365210    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:54:16.367782 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:54:16.367747    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed9790f_a654_42a8_9a3b_4d200585bb34.slice/crio-db47b9c8ca7690aa376d33820ddb0a5691bb3c27fc6a75f180f2c6ecba2881c5 WatchSource:0}: Error finding container db47b9c8ca7690aa376d33820ddb0a5691bb3c27fc6a75f180f2c6ecba2881c5: Status 404 returned error can't find the container with id db47b9c8ca7690aa376d33820ddb0a5691bb3c27fc6a75f180f2c6ecba2881c5
Sep 08 13:54:16.370048 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.370011    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:16.370146 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.370093    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:16.370215 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.370137    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:16.630576 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.630476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerStarted","Data":"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"}
Sep 08 13:54:16.630576 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.630538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerStarted","Data":"db47b9c8ca7690aa376d33820ddb0a5691bb3c27fc6a75f180f2c6ecba2881c5"}
Sep 08 13:54:16.650371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:16.650291    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podStartSLOduration=1.650272557 podStartE2EDuration="1.650272557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:16.648738001 +0000 UTC m=+1539.965141580" watchObservedRunningTime="2026-09-08 13:54:16.650272557 +0000 UTC m=+1539.966676110"
Sep 08 13:54:17.071620 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:17.071574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:17.073448 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:17.073421    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:17.073587 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:17.073467    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:18.076800 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:18.076769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:18.641696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:18.641664    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:18.642604 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:18.642582    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:20.466893 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.466857    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:54:20.470899 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.470876    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.473413 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.473390    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-569975-dockercfg-b7czx\""
Sep 08 13:54:20.473548 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.473422    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-569975\""
Sep 08 13:54:20.481940 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.481910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:54:20.487072 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.487043    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:54:20.491528 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.491510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.493445 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.493418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.493509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.493490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2cn\" (UniqueName: \"kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.503441 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.503415    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:54:20.595549 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.595329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd26b\" (UniqueName: \"kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.595549 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.595450    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.595549 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.595509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2cn\" (UniqueName: \"kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.595869 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.595568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.596421 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.596394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.607772 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.607739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2cn\" (UniqueName: \"kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn\") pod \"payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.697199 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.697156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd26b\" (UniqueName: \"kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.697428 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.697246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.697984 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.697960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.708947 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.708917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd26b\" (UniqueName: \"kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b\") pod \"payload-processing-e2e-api-a-569975-c4589d885-4j9pq\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") " pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:20.782886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.782796    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:20.803749 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:20.803722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:21.061785 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.061758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:54:21.064477 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:54:21.064445    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698aab8c_33a8_409a_98d5_51c59e3872db.slice/crio-1c89633c8c2d4880b5c3fbecdebbb1d2affa410a760cb391c41d1b0f5ae48c97 WatchSource:0}: Error finding container 1c89633c8c2d4880b5c3fbecdebbb1d2affa410a760cb391c41d1b0f5ae48c97: Status 404 returned error can't find the container with id 1c89633c8c2d4880b5c3fbecdebbb1d2affa410a760cb391c41d1b0f5ae48c97
Sep 08 13:54:21.092868 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.092839    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:54:21.097521 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:54:21.097488    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f6d2a3_30ee_4377_abbe_5695475ba626.slice/crio-f0932ecbba8c4592db94019f435cd594c7552fd9ad51f1fffbce770f300c1318 WatchSource:0}: Error finding container f0932ecbba8c4592db94019f435cd594c7552fd9ad51f1fffbce770f300c1318: Status 404 returned error can't find the container with id f0932ecbba8c4592db94019f435cd594c7552fd9ad51f1fffbce770f300c1318
Sep 08 13:54:21.662025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.661990    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" event={"ID":"698aab8c-33a8-409a-98d5-51c59e3872db","Type":"ContainerStarted","Data":"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"}
Sep 08 13:54:21.662532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.662055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" event={"ID":"698aab8c-33a8-409a-98d5-51c59e3872db","Type":"ContainerStarted","Data":"1c89633c8c2d4880b5c3fbecdebbb1d2affa410a760cb391c41d1b0f5ae48c97"}
Sep 08 13:54:21.662532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.662213    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:21.663974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.663946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" event={"ID":"11f6d2a3-30ee-4377-abbe-5695475ba626","Type":"ContainerStarted","Data":"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"}
Sep 08 13:54:21.664125 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.663996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" event={"ID":"11f6d2a3-30ee-4377-abbe-5695475ba626","Type":"ContainerStarted","Data":"f0932ecbba8c4592db94019f435cd594c7552fd9ad51f1fffbce770f300c1318"}
Sep 08 13:54:21.664125 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.664040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:21.677374 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.677290    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" podStartSLOduration=1.6772687579999999 podStartE2EDuration="1.677268758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:21.677055683 +0000 UTC m=+1544.993459239" watchObservedRunningTime="2026-09-08 13:54:21.677268758 +0000 UTC m=+1544.993672311"
Sep 08 13:54:21.694072 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:21.694020    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" podStartSLOduration=1.694001453 podStartE2EDuration="1.694001453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:21.69262253 +0000 UTC m=+1545.009026119" watchObservedRunningTime="2026-09-08 13:54:21.694001453 +0000 UTC m=+1545.010405056"
Sep 08 13:54:32.672389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:32.672331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:54:32.672834 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:32.672519    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:54:37.495497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:37.495452    2577 scope.go:117] "RemoveContainer" containerID="1a6a3371b6f9ade436444133685650e188b3c817ca0da3c77513aa0b6c344836"
Sep 08 13:54:43.788150 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.788066    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerID="269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c" exitCode=0
Sep 08 13:54:43.788948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.788142    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerDied","Data":"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"}
Sep 08 13:54:43.788948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.788588    2577 scope.go:117] "RemoveContainer" containerID="269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"
Sep 08 13:54:43.789739 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.789709    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:43.789845 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.789794    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:43.789915 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:43.789842    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:54:44.800042 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:44.799996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerStarted","Data":"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"}
Sep 08 13:54:45.072081 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:45.071998    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:45.073122 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:45.073097    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:45.073254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:45.073144    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:46.071968 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:46.071928    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:46.072416 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:46.072003    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:47.071445 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:47.071378    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:47.071729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:47.071499    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:48.071886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:48.071848    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:48.072272 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:48.071912    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:48.642269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:48.642229    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:49.071836 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.071732    2577 patch_prober.go:28] interesting pod/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 08 13:54:49.071836 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.071808    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 08 13:54:49.106592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.106544    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:54:49.109771 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.109742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.112126 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.112105    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-8ef041-dockercfg-94fj8\""
Sep 08 13:54:49.112126 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.112117    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-8ef041\""
Sep 08 13:54:49.116114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.116091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:54:49.211636 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.211596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l77z\" (UniqueName: \"kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.211805 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.211658    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.312484 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.312445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l77z\" (UniqueName: \"kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.312691 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.312667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.313596 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.313571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.324483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.324417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l77z\" (UniqueName: \"kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z\") pod \"payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.422442 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.422405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.699997 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.699939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:54:49.702573 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:54:49.702542    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd257aad_f052_4375_a88e_acd4a95a1d47.slice/crio-f3cb5e0eb206310c3621d2b0e407a32c79526d3c4104fe51063dc911ab72f4f7 WatchSource:0}: Error finding container f3cb5e0eb206310c3621d2b0e407a32c79526d3c4104fe51063dc911ab72f4f7: Status 404 returned error can't find the container with id f3cb5e0eb206310c3621d2b0e407a32c79526d3c4104fe51063dc911ab72f4f7
Sep 08 13:54:49.841482 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.841445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" event={"ID":"bd257aad-f052-4375-a88e-acd4a95a1d47","Type":"ContainerStarted","Data":"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"}
Sep 08 13:54:49.841696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.841505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" event={"ID":"bd257aad-f052-4375-a88e-acd4a95a1d47","Type":"ContainerStarted","Data":"f3cb5e0eb206310c3621d2b0e407a32c79526d3c4104fe51063dc911ab72f4f7"}
Sep 08 13:54:49.841696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.841532    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:54:49.859700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:49.859653    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" podStartSLOduration=0.859634822 podStartE2EDuration="859.634822ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:49.857047946 +0000 UTC m=+1573.173451543" watchObservedRunningTime="2026-09-08 13:54:49.859634822 +0000 UTC m=+1573.176038375"
Sep 08 13:54:50.076244 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:50.076165    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:54:50.077223 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:54:50.077207    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:55:00.849061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:00.849030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:55:31.183965 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.183907    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:55:31.185257 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.185223    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" podUID="11f6d2a3-30ee-4377-abbe-5695475ba626" containerName="payload-processing" containerID="cri-o://bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5" gracePeriod=30
Sep 08 13:55:31.189034 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.189006    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:55:31.189386 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.189293    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" podUID="698aab8c-33a8-409a-98d5-51c59e3872db" containerName="payload-pre-processing" containerID="cri-o://a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6" gracePeriod=30
Sep 08 13:55:31.487513 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.487491    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:55:31.490700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.490681    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:55:31.562152 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume\") pod \"11f6d2a3-30ee-4377-abbe-5695475ba626\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") "
Sep 08 13:55:31.562336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd2cn\" (UniqueName: \"kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn\") pod \"698aab8c-33a8-409a-98d5-51c59e3872db\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") "
Sep 08 13:55:31.562336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd26b\" (UniqueName: \"kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b\") pod \"11f6d2a3-30ee-4377-abbe-5695475ba626\" (UID: \"11f6d2a3-30ee-4377-abbe-5695475ba626\") "
Sep 08 13:55:31.562336 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume\") pod \"698aab8c-33a8-409a-98d5-51c59e3872db\" (UID: \"698aab8c-33a8-409a-98d5-51c59e3872db\") "
Sep 08 13:55:31.562646 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562559    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11f6d2a3-30ee-4377-abbe-5695475ba626" (UID: "11f6d2a3-30ee-4377-abbe-5695475ba626"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:31.562744 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562725    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11f6d2a3-30ee-4377-abbe-5695475ba626-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.562861 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.562836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "698aab8c-33a8-409a-98d5-51c59e3872db" (UID: "698aab8c-33a8-409a-98d5-51c59e3872db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:31.564616 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.564595    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b" (OuterVolumeSpecName: "kube-api-access-hd26b") pod "11f6d2a3-30ee-4377-abbe-5695475ba626" (UID: "11f6d2a3-30ee-4377-abbe-5695475ba626"). InnerVolumeSpecName "kube-api-access-hd26b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:31.564709 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.564626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn" (OuterVolumeSpecName: "kube-api-access-wd2cn") pod "698aab8c-33a8-409a-98d5-51c59e3872db" (UID: "698aab8c-33a8-409a-98d5-51c59e3872db"). InnerVolumeSpecName "kube-api-access-wd2cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:31.663811 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.663772    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd2cn\" (UniqueName: \"kubernetes.io/projected/698aab8c-33a8-409a-98d5-51c59e3872db-kube-api-access-wd2cn\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.663811 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.663805    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd26b\" (UniqueName: \"kubernetes.io/projected/11f6d2a3-30ee-4377-abbe-5695475ba626-kube-api-access-hd26b\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.663811 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:31.663816    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/698aab8c-33a8-409a-98d5-51c59e3872db-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:32.069869 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.069830    2577 generic.go:356] "Generic (PLEG): container finished" podID="11f6d2a3-30ee-4377-abbe-5695475ba626" containerID="bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5" exitCode=0
Sep 08 13:55:32.070061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.069878    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"
Sep 08 13:55:32.070061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.069909    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" event={"ID":"11f6d2a3-30ee-4377-abbe-5695475ba626","Type":"ContainerDied","Data":"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"}
Sep 08 13:55:32.070061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.069961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq" event={"ID":"11f6d2a3-30ee-4377-abbe-5695475ba626","Type":"ContainerDied","Data":"f0932ecbba8c4592db94019f435cd594c7552fd9ad51f1fffbce770f300c1318"}
Sep 08 13:55:32.070061 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.069991    2577 scope.go:117] "RemoveContainer" containerID="bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"
Sep 08 13:55:32.071675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.071326    2577 generic.go:356] "Generic (PLEG): container finished" podID="698aab8c-33a8-409a-98d5-51c59e3872db" containerID="a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6" exitCode=0
Sep 08 13:55:32.071675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.071378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" event={"ID":"698aab8c-33a8-409a-98d5-51c59e3872db","Type":"ContainerDied","Data":"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"}
Sep 08 13:55:32.071675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.071438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r" event={"ID":"698aab8c-33a8-409a-98d5-51c59e3872db","Type":"ContainerDied","Data":"1c89633c8c2d4880b5c3fbecdebbb1d2affa410a760cb391c41d1b0f5ae48c97"}
Sep 08 13:55:32.071675 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.071394    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"
Sep 08 13:55:32.081816 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.081793    2577 scope.go:117] "RemoveContainer" containerID="bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"
Sep 08 13:55:32.082260 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:55:32.082118    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5\": container with ID starting with bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5 not found: ID does not exist" containerID="bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"
Sep 08 13:55:32.082260 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.082151    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5"} err="failed to get container status \"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5\": rpc error: code = NotFound desc = could not find container \"bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5\": container with ID starting with bc8db3c23c92aae7650b201caf0c1450f84a8c15b6a3047852df419561633dc5 not found: ID does not exist"
Sep 08 13:55:32.082260 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.082177    2577 scope.go:117] "RemoveContainer" containerID="a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"
Sep 08 13:55:32.090668 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.090648    2577 scope.go:117] "RemoveContainer" containerID="a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"
Sep 08 13:55:32.090960 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:55:32.090940    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6\": container with ID starting with a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6 not found: ID does not exist" containerID="a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"
Sep 08 13:55:32.091029 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.090969    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6"} err="failed to get container status \"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6\": rpc error: code = NotFound desc = could not find container \"a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6\": container with ID starting with a9e6efe65563b4b3c9931cd172c89d8fbecdf92e52b750a1cf074460795d49e6 not found: ID does not exist"
Sep 08 13:55:32.094612 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.094590    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:55:32.098526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.098502    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-569975-c4589d885-4j9pq"]
Sep 08 13:55:32.107693 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.107664    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:55:32.112149 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:32.112125    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-569975-76d8db5cfd-zsr8r"]
Sep 08 13:55:33.214224 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:33.214194    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11f6d2a3-30ee-4377-abbe-5695475ba626" path="/var/lib/kubelet/pods/11f6d2a3-30ee-4377-abbe-5695475ba626/volumes"
Sep 08 13:55:33.214594 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:33.214583    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698aab8c-33a8-409a-98d5-51c59e3872db" path="/var/lib/kubelet/pods/698aab8c-33a8-409a-98d5-51c59e3872db/volumes"
Sep 08 13:55:37.520727 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:37.520699    2577 scope.go:117] "RemoveContainer" containerID="79b562c39b56b748dc5f891e7a480fbc8914a96e1e3b4ed0daf9159fce553c3a"
Sep 08 13:55:37.529518 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:37.529501    2577 scope.go:117] "RemoveContainer" containerID="72042cf11a6790a1ffac744ddaaa604dc2033f68b3a75857cc8fe55ebf0d04d1"
Sep 08 13:55:38.080779 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.080743    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:55:38.081101 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.081070    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy" containerID="cri-o://33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6" gracePeriod=30
Sep 08 13:55:38.431196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431154    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"]
Sep 08 13:55:38.431619 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431604    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11f6d2a3-30ee-4377-abbe-5695475ba626" containerName="payload-processing"
Sep 08 13:55:38.431619 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431618    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11f6d2a3-30ee-4377-abbe-5695475ba626" containerName="payload-processing"
Sep 08 13:55:38.431797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431638    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698aab8c-33a8-409a-98d5-51c59e3872db" containerName="payload-pre-processing"
Sep 08 13:55:38.431797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431644    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="698aab8c-33a8-409a-98d5-51c59e3872db" containerName="payload-pre-processing"
Sep 08 13:55:38.431797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431705    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11f6d2a3-30ee-4377-abbe-5695475ba626" containerName="payload-processing"
Sep 08 13:55:38.431797 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.431714    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="698aab8c-33a8-409a-98d5-51c59e3872db" containerName="payload-pre-processing"
Sep 08 13:55:38.436113 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.436088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:38.438483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.438455    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:55:38.446298 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.446273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"]
Sep 08 13:55:38.530035 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.529999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8ef041-m4hts\" (UID: \"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:38.631114 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.631080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8ef041-m4hts\" (UID: \"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:38.631760 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.631742    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-8ef041-m4hts\" (UID: \"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:38.748155 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:38.748055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:39.023538 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:39.023452    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"]
Sep 08 13:55:39.028519 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:55:39.028467    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dabb8ad_23d6_418e_b5bc_b3043e1a91bf.slice/crio-f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe WatchSource:0}: Error finding container f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe: Status 404 returned error can't find the container with id f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe
Sep 08 13:55:39.116032 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:39.115991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts" event={"ID":"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf","Type":"ContainerStarted","Data":"f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe"}
Sep 08 13:55:40.122866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:40.122775    2577 generic.go:356] "Generic (PLEG): container finished" podID="2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" containerID="5d784bcb82bc1762ef951a02d47f3e64449c1386b9aaeb5883ab9a50a88aeb18" exitCode=0
Sep 08 13:55:40.122866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:40.122851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts" event={"ID":"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf","Type":"ContainerDied","Data":"5d784bcb82bc1762ef951a02d47f3e64449c1386b9aaeb5883ab9a50a88aeb18"}
Sep 08 13:55:41.264424 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:41.264394    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:41.358385 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:41.358335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca\") pod \"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf\" (UID: \"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf\") "
Sep 08 13:55:41.358684 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:41.358661    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" (UID: "2dabb8ad-23d6-418e-b5bc-b3043e1a91bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:41.460026 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:41.459996    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:42.134447 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:42.134423    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"
Sep 08 13:55:42.134606 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:42.134456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts" event={"ID":"2dabb8ad-23d6-418e-b5bc-b3043e1a91bf","Type":"ContainerDied","Data":"f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe"}
Sep 08 13:55:42.134606 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:42.134496    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2e95a2d9713fc72d56e6a10526d84d0cdb373c6a1ff08efe2a405406904b5fe"
Sep 08 13:55:43.346592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.346520    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:55:43.480478 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480436    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480598    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480623    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480678 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480732    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp6pn\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480788    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy\") pod \"6ed9790f-a654-42a8-9a3b-4d200585bb34\" (UID: \"6ed9790f-a654-42a8-9a3b-4d200585bb34\") "
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480879    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:55:43.480944 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.480899    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:55:43.481253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481083    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:55:43.481253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481232    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.481253 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481252    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.481409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481265    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.481409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481278    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.481409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.481227    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:43.483145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.483123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn" (OuterVolumeSpecName: "kube-api-access-cp6pn") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "kube-api-access-cp6pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:43.483145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.483131    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 13:55:43.483453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.483428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:43.483578 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.483454    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ed9790f-a654-42a8-9a3b-4d200585bb34" (UID: "6ed9790f-a654-42a8-9a3b-4d200585bb34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:55:43.581865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.581829    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.581865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.581859    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.581865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.581870    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp6pn\" (UniqueName: \"kubernetes.io/projected/6ed9790f-a654-42a8-9a3b-4d200585bb34-kube-api-access-cp6pn\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.581865 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.581879    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ed9790f-a654-42a8-9a3b-4d200585bb34-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:43.582134 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:43.581887    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ed9790f-a654-42a8-9a3b-4d200585bb34-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:44.148770 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.148740    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerID="33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6" exitCode=0
Sep 08 13:55:44.148949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.148795    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"
Sep 08 13:55:44.148949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.148829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerDied","Data":"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"}
Sep 08 13:55:44.148949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.148890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b" event={"ID":"6ed9790f-a654-42a8-9a3b-4d200585bb34","Type":"ContainerDied","Data":"db47b9c8ca7690aa376d33820ddb0a5691bb3c27fc6a75f180f2c6ecba2881c5"}
Sep 08 13:55:44.148949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.148919    2577 scope.go:117] "RemoveContainer" containerID="33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"
Sep 08 13:55:44.159072 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.159053    2577 scope.go:117] "RemoveContainer" containerID="269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"
Sep 08 13:55:44.168163 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.168143    2577 scope.go:117] "RemoveContainer" containerID="33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"
Sep 08 13:55:44.168499 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:55:44.168477    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6\": container with ID starting with 33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6 not found: ID does not exist" containerID="33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"
Sep 08 13:55:44.168591 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.168507    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6"} err="failed to get container status \"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6\": rpc error: code = NotFound desc = could not find container \"33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6\": container with ID starting with 33f45f1c14c5b8dd37ca2ea642e59fac85cb6ab4be7863a13817a24318336da6 not found: ID does not exist"
Sep 08 13:55:44.168591 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.168526    2577 scope.go:117] "RemoveContainer" containerID="269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"
Sep 08 13:55:44.168840 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:55:44.168823    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c\": container with ID starting with 269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c not found: ID does not exist" containerID="269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"
Sep 08 13:55:44.168883 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.168850    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c"} err="failed to get container status \"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c\": rpc error: code = NotFound desc = could not find container \"269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c\": container with ID starting with 269bc37143379f9d68c80de6e1106316f6eb9cde5e8a29a8bf73ee08f275634c not found: ID does not exist"
Sep 08 13:55:44.170245 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.170228    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:55:44.177020 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:44.176995    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-569975-openshift-default-5f68c88b46-q2h4b"]
Sep 08 13:55:45.213299 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:45.213267    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" path="/var/lib/kubelet/pods/6ed9790f-a654-42a8-9a3b-4d200585bb34/volumes"
Sep 08 13:55:54.132396 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.132361    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:55:54.132829 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.132577    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" podUID="bd257aad-f052-4375-a88e-acd4a95a1d47" containerName="payload-pre-processing" containerID="cri-o://9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70" gracePeriod=30
Sep 08 13:55:54.302145 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.302114    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"]
Sep 08 13:55:54.306650 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.306624    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-8ef041-m4hts"]
Sep 08 13:55:54.390116 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.390090    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:55:54.491585 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.491552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume\") pod \"bd257aad-f052-4375-a88e-acd4a95a1d47\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") "
Sep 08 13:55:54.491757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.491713    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l77z\" (UniqueName: \"kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z\") pod \"bd257aad-f052-4375-a88e-acd4a95a1d47\" (UID: \"bd257aad-f052-4375-a88e-acd4a95a1d47\") "
Sep 08 13:55:54.491959 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.491932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bd257aad-f052-4375-a88e-acd4a95a1d47" (UID: "bd257aad-f052-4375-a88e-acd4a95a1d47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:54.493949 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.493918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z" (OuterVolumeSpecName: "kube-api-access-2l77z") pod "bd257aad-f052-4375-a88e-acd4a95a1d47" (UID: "bd257aad-f052-4375-a88e-acd4a95a1d47"). InnerVolumeSpecName "kube-api-access-2l77z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:54.593034 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.592988    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l77z\" (UniqueName: \"kubernetes.io/projected/bd257aad-f052-4375-a88e-acd4a95a1d47-kube-api-access-2l77z\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:54.593034 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:54.593029    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd257aad-f052-4375-a88e-acd4a95a1d47-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:55:55.210773 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.210738    2577 generic.go:356] "Generic (PLEG): container finished" podID="bd257aad-f052-4375-a88e-acd4a95a1d47" containerID="9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70" exitCode=0
Sep 08 13:55:55.211125 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.210857    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"
Sep 08 13:55:55.213482 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.213456    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" path="/var/lib/kubelet/pods/2dabb8ad-23d6-418e-b5bc-b3043e1a91bf/volumes"
Sep 08 13:55:55.213836 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.213816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" event={"ID":"bd257aad-f052-4375-a88e-acd4a95a1d47","Type":"ContainerDied","Data":"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"}
Sep 08 13:55:55.213919 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.213856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9" event={"ID":"bd257aad-f052-4375-a88e-acd4a95a1d47","Type":"ContainerDied","Data":"f3cb5e0eb206310c3621d2b0e407a32c79526d3c4104fe51063dc911ab72f4f7"}
Sep 08 13:55:55.213919 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.213877    2577 scope.go:117] "RemoveContainer" containerID="9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"
Sep 08 13:55:55.222829 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.222813    2577 scope.go:117] "RemoveContainer" containerID="9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"
Sep 08 13:55:55.223104 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:55:55.223085    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70\": container with ID starting with 9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70 not found: ID does not exist" containerID="9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"
Sep 08 13:55:55.223153 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.223113    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70"} err="failed to get container status \"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70\": rpc error: code = NotFound desc = could not find container \"9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70\": container with ID starting with 9179b4b2cbc954f25758fa7e98c42198d0b56e996fb03f14c3a1a4995cb3ef70 not found: ID does not exist"
Sep 08 13:55:55.234815 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.234788    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:55:55.240098 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:55.240078    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8ef041-8548b6979d-d87m9"]
Sep 08 13:55:57.213772 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:55:57.213740    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd257aad-f052-4375-a88e-acd4a95a1d47" path="/var/lib/kubelet/pods/bd257aad-f052-4375-a88e-acd4a95a1d47/volumes"
Sep 08 13:56:01.565879 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.565844    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"]
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566297    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd257aad-f052-4375-a88e-acd4a95a1d47" containerName="payload-pre-processing"
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566312    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd257aad-f052-4375-a88e-acd4a95a1d47" containerName="payload-pre-processing"
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566330    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566336    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566354    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" containerName="revoke-keys"
Sep 08 13:56:01.566364 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566363    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" containerName="revoke-keys"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566380    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566386    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566461    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566468    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed9790f-a654-42a8-9a3b-4d200585bb34" containerName="istio-proxy"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566475    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dabb8ad-23d6-418e-b5bc-b3043e1a91bf" containerName="revoke-keys"
Sep 08 13:56:01.566564 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.566482    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd257aad-f052-4375-a88e-acd4a95a1d47" containerName="payload-pre-processing"
Sep 08 13:56:01.571236 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.571217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:01.573268 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.573239    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:56:01.577419 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.577390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"]
Sep 08 13:56:01.657670 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.657631    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl\" (UID: \"8efdb492-7856-4207-b831-0aa560272616\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:01.758165 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.758126    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl\" (UID: \"8efdb492-7856-4207-b831-0aa560272616\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:01.758747 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.758726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl\" (UID: \"8efdb492-7856-4207-b831-0aa560272616\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:01.885184 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:01.885163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:02.164731 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:02.164706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"]
Sep 08 13:56:02.166857 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:02.166831    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efdb492_7856_4207_b831_0aa560272616.slice/crio-9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85 WatchSource:0}: Error finding container 9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85: Status 404 returned error can't find the container with id 9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85
Sep 08 13:56:02.249524 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:02.249491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl" event={"ID":"8efdb492-7856-4207-b831-0aa560272616","Type":"ContainerStarted","Data":"9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85"}
Sep 08 13:56:03.256425 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:03.256307    2577 generic.go:356] "Generic (PLEG): container finished" podID="8efdb492-7856-4207-b831-0aa560272616" containerID="fec05674ba9eac30da3c7aff369703eb2e3fee766ababd1dba52f9d7c6dc3298" exitCode=0
Sep 08 13:56:03.256425 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:03.256389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl" event={"ID":"8efdb492-7856-4207-b831-0aa560272616","Type":"ContainerDied","Data":"fec05674ba9eac30da3c7aff369703eb2e3fee766ababd1dba52f9d7c6dc3298"}
Sep 08 13:56:04.398669 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:04.398641    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:04.483662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:04.483603    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca\") pod \"8efdb492-7856-4207-b831-0aa560272616\" (UID: \"8efdb492-7856-4207-b831-0aa560272616\") "
Sep 08 13:56:04.483986 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:04.483962    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8efdb492-7856-4207-b831-0aa560272616" (UID: "8efdb492-7856-4207-b831-0aa560272616"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:04.585120 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:04.585031    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8efdb492-7856-4207-b831-0aa560272616-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:05.268554 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:05.268524    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"
Sep 08 13:56:05.268747 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:05.268520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl" event={"ID":"8efdb492-7856-4207-b831-0aa560272616","Type":"ContainerDied","Data":"9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85"}
Sep 08 13:56:05.268747 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:05.268649    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e470042f828deab6a86372ff794d41c2942c852568dd91a79808b2d36dada85"
Sep 08 13:56:12.219405 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.219373    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:56:12.219977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.219565    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" podUID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" containerName="payload-processing" containerID="cri-o://ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879" gracePeriod=30
Sep 08 13:56:12.431790 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.431757    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"]
Sep 08 13:56:12.433866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.433841    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-3c9fc8-9gwzl"]
Sep 08 13:56:12.481170 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.481109    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:56:12.560803 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.560775    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkfpv\" (UniqueName: \"kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv\") pod \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") "
Sep 08 13:56:12.560969 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.560831    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume\") pod \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\" (UID: \"050ee39f-4fd4-498a-8215-dc8898fe0bc8\") "
Sep 08 13:56:12.561222 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.561193    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "050ee39f-4fd4-498a-8215-dc8898fe0bc8" (UID: "050ee39f-4fd4-498a-8215-dc8898fe0bc8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:12.562992 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.562972    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv" (OuterVolumeSpecName: "kube-api-access-pkfpv") pod "050ee39f-4fd4-498a-8215-dc8898fe0bc8" (UID: "050ee39f-4fd4-498a-8215-dc8898fe0bc8"). InnerVolumeSpecName "kube-api-access-pkfpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:12.662178 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.662147    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050ee39f-4fd4-498a-8215-dc8898fe0bc8-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:12.662178 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:12.662174    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkfpv\" (UniqueName: \"kubernetes.io/projected/050ee39f-4fd4-498a-8215-dc8898fe0bc8-kube-api-access-pkfpv\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:13.213394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.213333    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8efdb492-7856-4207-b831-0aa560272616" path="/var/lib/kubelet/pods/8efdb492-7856-4207-b831-0aa560272616/volumes"
Sep 08 13:56:13.310177 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.310145    2577 generic.go:356] "Generic (PLEG): container finished" podID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" containerID="ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879" exitCode=0
Sep 08 13:56:13.310673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.310196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" event={"ID":"050ee39f-4fd4-498a-8215-dc8898fe0bc8","Type":"ContainerDied","Data":"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"}
Sep 08 13:56:13.310673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.310224    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"
Sep 08 13:56:13.310673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.310239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj" event={"ID":"050ee39f-4fd4-498a-8215-dc8898fe0bc8","Type":"ContainerDied","Data":"b3945ba91b5f2fa94bb690bc4396331ee43704b4f57bfb76363cc4404d3ced91"}
Sep 08 13:56:13.310673 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.310259    2577 scope.go:117] "RemoveContainer" containerID="ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"
Sep 08 13:56:13.319672 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.319647    2577 scope.go:117] "RemoveContainer" containerID="ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"
Sep 08 13:56:13.320003 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:56:13.319979    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879\": container with ID starting with ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879 not found: ID does not exist" containerID="ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"
Sep 08 13:56:13.320065 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.320012    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879"} err="failed to get container status \"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879\": rpc error: code = NotFound desc = could not find container \"ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879\": container with ID starting with ae5c75a41866663aaf9cfd7ab27075c2b3263d73781036f2c6fa7a640e565879 not found: ID does not exist"
Sep 08 13:56:13.334708 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.334661    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:56:13.336599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:13.336574    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-3c9fc8-5dcc55f84f-8rdzj"]
Sep 08 13:56:15.213504 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:15.213470    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" path="/var/lib/kubelet/pods/050ee39f-4fd4-498a-8215-dc8898fe0bc8/volumes"
Sep 08 13:56:16.250509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250471    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 13:56:16.250896 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250882    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efdb492-7856-4207-b831-0aa560272616" containerName="revoke-keys"
Sep 08 13:56:16.250896 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250894    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efdb492-7856-4207-b831-0aa560272616" containerName="revoke-keys"
Sep 08 13:56:16.250988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250906    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" containerName="payload-processing"
Sep 08 13:56:16.250988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250912    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" containerName="payload-processing"
Sep 08 13:56:16.250988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250973    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="050ee39f-4fd4-498a-8215-dc8898fe0bc8" containerName="payload-processing"
Sep 08 13:56:16.250988 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.250984    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efdb492-7856-4207-b831-0aa560272616" containerName="revoke-keys"
Sep 08 13:56:16.255456 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.255432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.257720 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.257700    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-2199d0-openshift-default-dockercfg-nj6c4\""
Sep 08 13:56:16.265754 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.265731    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 13:56:16.298181 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298193    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db4hw\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298551 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298551 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298551 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.298551 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.298519    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399705 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399670    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db4hw\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.399884 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400156 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400156 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399926    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400156 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.399959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400311 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.400249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400311 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.400286    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400497 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.400476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400561 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.400519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.400784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.400682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.402235 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.402214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.402409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.402392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.408170 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.408146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db4hw\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.408273 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.408182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token\") pod \"e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") " pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.567148 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.567056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:16.844301 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.844265    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 13:56:16.847294 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:16.847266    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1929a2_fcac_4264_9420_24643f9c0628.slice/crio-34e6dc6c330a7f325a6b9796179da8a4b6ddb79fa5a99ec9618ff0e3d23f41a4 WatchSource:0}: Error finding container 34e6dc6c330a7f325a6b9796179da8a4b6ddb79fa5a99ec9618ff0e3d23f41a4: Status 404 returned error can't find the container with id 34e6dc6c330a7f325a6b9796179da8a4b6ddb79fa5a99ec9618ff0e3d23f41a4
Sep 08 13:56:16.849641 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.849609    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:16.849706 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.849678    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:16.849745 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:16.849706    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:17.336006 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:17.335969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerStarted","Data":"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"}
Sep 08 13:56:17.336453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:17.336020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerStarted","Data":"34e6dc6c330a7f325a6b9796179da8a4b6ddb79fa5a99ec9618ff0e3d23f41a4"}
Sep 08 13:56:17.357158 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:17.357102    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podStartSLOduration=1.357085566 podStartE2EDuration="1.357085566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:17.354065583 +0000 UTC m=+1660.670469149" watchObservedRunningTime="2026-09-08 13:56:17.357085566 +0000 UTC m=+1660.673489119"
Sep 08 13:56:17.568056 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:17.568019    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:17.573635 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:17.573605    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:18.340962 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:18.340929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:18.341922 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:18.341904    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:19.118154 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.118122    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:56:19.118424 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.118395    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" containerID="cri-o://56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66" gracePeriod=30
Sep 08 13:56:19.557481 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.557447    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"]
Sep 08 13:56:19.561404 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.561388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:19.564287 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.564258    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:56:19.577304 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.577275    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"]
Sep 08 13:56:19.634370 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.634309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2\" (UID: \"4ea4e340-ac55-42bd-9a3e-732b49214a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:19.735246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.735208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2\" (UID: \"4ea4e340-ac55-42bd-9a3e-732b49214a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:19.735875 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.735854    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2\" (UID: \"4ea4e340-ac55-42bd-9a3e-732b49214a6e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:19.817286 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.817194    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 13:56:19.817286 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.817261    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:56:19.872209 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:19.872177    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:20.149465 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:20.149414    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"]
Sep 08 13:56:20.151715 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:20.151689    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea4e340_ac55_42bd_9a3e_732b49214a6e.slice/crio-2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb WatchSource:0}: Error finding container 2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb: Status 404 returned error can't find the container with id 2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb
Sep 08 13:56:20.352873 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:20.352842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2" event={"ID":"4ea4e340-ac55-42bd-9a3e-732b49214a6e","Type":"ContainerStarted","Data":"2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb"}
Sep 08 13:56:21.359698 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.359610    2577 generic.go:356] "Generic (PLEG): container finished" podID="4ea4e340-ac55-42bd-9a3e-732b49214a6e" containerID="506f4cbf0364191c5a3e9301db03168eb668d8bcc94144176c65b7e62e72882a" exitCode=0
Sep 08 13:56:21.360078 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.359696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2" event={"ID":"4ea4e340-ac55-42bd-9a3e-732b49214a6e","Type":"ContainerDied","Data":"506f4cbf0364191c5a3e9301db03168eb668d8bcc94144176c65b7e62e72882a"}
Sep 08 13:56:21.376692 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.376662    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:56:21.380321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.380306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.382729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.382706    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-2199d0-dockercfg-wzbx4\""
Sep 08 13:56:21.383317 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.383301    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-2199d0\""
Sep 08 13:56:21.390803 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.390780    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:56:21.395414 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.395390    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:56:21.399237 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.399217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.409996 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.409972    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:56:21.452793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.452755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.452793 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.452793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clb2g\" (UniqueName: \"kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.453041 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.452871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52lz8\" (UniqueName: \"kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.453104 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.453069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.554434 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.554396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.554631 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.554442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.554631 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.554461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clb2g\" (UniqueName: \"kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.554631 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.554512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52lz8\" (UniqueName: \"kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.555150 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.555125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.555239 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.555160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.564014 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.563982    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clb2g\" (UniqueName: \"kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g\") pod \"payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.564014 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.563997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52lz8\" (UniqueName: \"kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8\") pod \"payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.694039 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.694005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:21.711035 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.711004    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:21.991826 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:21.991795    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:56:21.993699 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:21.993670    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9643ee2_2bff_463a_8464_607dec62a62b.slice/crio-fff2bfd087dfb346f3d8e4429063342afc6eaebae0369cf5e0836857e42da775 WatchSource:0}: Error finding container fff2bfd087dfb346f3d8e4429063342afc6eaebae0369cf5e0836857e42da775: Status 404 returned error can't find the container with id fff2bfd087dfb346f3d8e4429063342afc6eaebae0369cf5e0836857e42da775
Sep 08 13:56:22.010033 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.010007    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:56:22.013572 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:22.013540    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc3c6c0_6fff_427a_8103_48895e350e37.slice/crio-f82f15a98bd07447f6919c8fb7a5e97a0ecfcb550e1d2f7ef8de995d62582cb7 WatchSource:0}: Error finding container f82f15a98bd07447f6919c8fb7a5e97a0ecfcb550e1d2f7ef8de995d62582cb7: Status 404 returned error can't find the container with id f82f15a98bd07447f6919c8fb7a5e97a0ecfcb550e1d2f7ef8de995d62582cb7
Sep 08 13:56:22.367133 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.367027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" event={"ID":"f9643ee2-2bff-463a-8464-607dec62a62b","Type":"ContainerStarted","Data":"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"}
Sep 08 13:56:22.367133 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.367091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" event={"ID":"f9643ee2-2bff-463a-8464-607dec62a62b","Type":"ContainerStarted","Data":"fff2bfd087dfb346f3d8e4429063342afc6eaebae0369cf5e0836857e42da775"}
Sep 08 13:56:22.367666 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.367205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:22.368899 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.368874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" event={"ID":"cdc3c6c0-6fff-427a-8103-48895e350e37","Type":"ContainerStarted","Data":"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"}
Sep 08 13:56:22.369009 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.368911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" event={"ID":"cdc3c6c0-6fff-427a-8103-48895e350e37","Type":"ContainerStarted","Data":"f82f15a98bd07447f6919c8fb7a5e97a0ecfcb550e1d2f7ef8de995d62582cb7"}
Sep 08 13:56:22.386395 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.386313    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" podStartSLOduration=1.386292146 podStartE2EDuration="1.386292146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:22.385668534 +0000 UTC m=+1665.702072103" watchObservedRunningTime="2026-09-08 13:56:22.386292146 +0000 UTC m=+1665.702695701"
Sep 08 13:56:22.406574 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.405176    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" podStartSLOduration=1.4051536869999999 podStartE2EDuration="1.405153687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:22.40474073 +0000 UTC m=+1665.721144299" watchObservedRunningTime="2026-09-08 13:56:22.405153687 +0000 UTC m=+1665.721557243"
Sep 08 13:56:22.524377 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.519949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:22.666723 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.666678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca\") pod \"4ea4e340-ac55-42bd-9a3e-732b49214a6e\" (UID: \"4ea4e340-ac55-42bd-9a3e-732b49214a6e\") "
Sep 08 13:56:22.667033 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.667011    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ea4e340-ac55-42bd-9a3e-732b49214a6e" (UID: "4ea4e340-ac55-42bd-9a3e-732b49214a6e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:22.767624 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:22.767587    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ea4e340-ac55-42bd-9a3e-732b49214a6e-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:23.375389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:23.375359    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"
Sep 08 13:56:23.375389 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:23.375335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2" event={"ID":"4ea4e340-ac55-42bd-9a3e-732b49214a6e","Type":"ContainerDied","Data":"2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb"}
Sep 08 13:56:23.375819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:23.375429    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a3078e262a64bb7f8e959ee80b4690ab54794b753aa34fc5dec60f2a46ed8cb"
Sep 08 13:56:23.375819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:23.375699    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:24.371331 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.371310    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:56:24.382939 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.382863    2577 generic.go:356] "Generic (PLEG): container finished" podID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerID="56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66" exitCode=0
Sep 08 13:56:24.382939 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.382911    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"
Sep 08 13:56:24.383373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.382941    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerDied","Data":"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"}
Sep 08 13:56:24.383373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.383003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg" event={"ID":"9ac385d3-571f-4d16-86c4-9562c6a73223","Type":"ContainerDied","Data":"ae5117d5484f3bcdaaa1be1754291180053ff685024df9fd86f0c96ecfbdd8ce"}
Sep 08 13:56:24.383373 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.383036    2577 scope.go:117] "RemoveContainer" containerID="56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"
Sep 08 13:56:24.393925 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.393784    2577 scope.go:117] "RemoveContainer" containerID="cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"
Sep 08 13:56:24.405932 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.405908    2577 scope.go:117] "RemoveContainer" containerID="56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"
Sep 08 13:56:24.406302 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:56:24.406280    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66\": container with ID starting with 56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66 not found: ID does not exist" containerID="56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"
Sep 08 13:56:24.406453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.406308    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66"} err="failed to get container status \"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66\": rpc error: code = NotFound desc = could not find container \"56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66\": container with ID starting with 56be6430dd9213f2c4eede030d3479deb17468defbc094eba1606ab5d74e4d66 not found: ID does not exist"
Sep 08 13:56:24.406453 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.406329    2577 scope.go:117] "RemoveContainer" containerID="cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"
Sep 08 13:56:24.406633 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:56:24.406617    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275\": container with ID starting with cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275 not found: ID does not exist" containerID="cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"
Sep 08 13:56:24.406694 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.406640    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275"} err="failed to get container status \"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275\": rpc error: code = NotFound desc = could not find container \"cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275\": container with ID starting with cc3df67ea7436cdfe4f5cbf437b7363428fda5f0441a7de12beb064a0d331275 not found: ID does not exist"
Sep 08 13:56:24.484311 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2s5t\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484319    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484403    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484509 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484670    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484757 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484783    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data\") pod \"9ac385d3-571f-4d16-86c4-9562c6a73223\" (UID: \"9ac385d3-571f-4d16-86c4-9562c6a73223\") "
Sep 08 13:56:24.484866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484810    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:24.484970 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.484915    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:24.485052 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:24.485629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data" (OuterVolumeSpecName: "istio-data") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:24.485629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485323    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ac385d3-571f-4d16-86c4-9562c6a73223-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.485629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485374    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.485629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485389    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.485629 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.485439    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:24.487013 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.486990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 13:56:24.487240 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.487217    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:24.487399 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.487334    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t" (OuterVolumeSpecName: "kube-api-access-w2s5t") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "kube-api-access-w2s5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:24.487528 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.487398    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ac385d3-571f-4d16-86c4-9562c6a73223" (UID: "9ac385d3-571f-4d16-86c4-9562c6a73223"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:24.586101 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586059    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.586101 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586095    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2s5t\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-kube-api-access-w2s5t\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.586101 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586109    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.586369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586120    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ac385d3-571f-4d16-86c4-9562c6a73223-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.586369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586132    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.586369 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.586142    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ac385d3-571f-4d16-86c4-9562c6a73223-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:24.706171 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.706134    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:56:24.711871 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:24.711844    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-3c9fc8-openshift-default-559645d994-qbdtg"]
Sep 08 13:56:25.214479 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:25.214444    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" path="/var/lib/kubelet/pods/9ac385d3-571f-4d16-86c4-9562c6a73223/volumes"
Sep 08 13:56:30.535521 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.535412    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:56:30.535977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.535679    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" podUID="803c5489-a1df-41e6-ab48-399ccab42ddb" containerName="payload-pre-processing" containerID="cri-o://0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994" gracePeriod=30
Sep 08 13:56:30.719053 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.719016    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"]
Sep 08 13:56:30.720982 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.720959    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d9970e-tmmt2"]
Sep 08 13:56:30.852318 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.852295    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:56:30.949599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.949557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxtdx\" (UniqueName: \"kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx\") pod \"803c5489-a1df-41e6-ab48-399ccab42ddb\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") "
Sep 08 13:56:30.949599 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.949597    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume\") pod \"803c5489-a1df-41e6-ab48-399ccab42ddb\" (UID: \"803c5489-a1df-41e6-ab48-399ccab42ddb\") "
Sep 08 13:56:30.949958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.949933    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "803c5489-a1df-41e6-ab48-399ccab42ddb" (UID: "803c5489-a1df-41e6-ab48-399ccab42ddb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:30.951792 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:30.951771    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx" (OuterVolumeSpecName: "kube-api-access-qxtdx") pod "803c5489-a1df-41e6-ab48-399ccab42ddb" (UID: "803c5489-a1df-41e6-ab48-399ccab42ddb"). InnerVolumeSpecName "kube-api-access-qxtdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:31.050700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.050658    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxtdx\" (UniqueName: \"kubernetes.io/projected/803c5489-a1df-41e6-ab48-399ccab42ddb-kube-api-access-qxtdx\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:31.050700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.050693    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/803c5489-a1df-41e6-ab48-399ccab42ddb-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:56:31.213586 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.213550    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ea4e340-ac55-42bd-9a3e-732b49214a6e" path="/var/lib/kubelet/pods/4ea4e340-ac55-42bd-9a3e-732b49214a6e/volumes"
Sep 08 13:56:31.427511 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.427481    2577 generic.go:356] "Generic (PLEG): container finished" podID="803c5489-a1df-41e6-ab48-399ccab42ddb" containerID="0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994" exitCode=0
Sep 08 13:56:31.427701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.427543    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"
Sep 08 13:56:31.427701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.427542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" event={"ID":"803c5489-a1df-41e6-ab48-399ccab42ddb","Type":"ContainerDied","Data":"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"}
Sep 08 13:56:31.427701 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.427663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9" event={"ID":"803c5489-a1df-41e6-ab48-399ccab42ddb","Type":"ContainerDied","Data":"91c3cbc7e64dc2ba857c359dcae3e5a88faf8842f6d2f20d9a704d5cbbbc5f4e"}
Sep 08 13:56:31.427864 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.427706    2577 scope.go:117] "RemoveContainer" containerID="0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"
Sep 08 13:56:31.437017 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.437002    2577 scope.go:117] "RemoveContainer" containerID="0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"
Sep 08 13:56:31.437278 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:56:31.437258    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994\": container with ID starting with 0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994 not found: ID does not exist" containerID="0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"
Sep 08 13:56:31.437327 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.437287    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994"} err="failed to get container status \"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994\": rpc error: code = NotFound desc = could not find container \"0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994\": container with ID starting with 0fc218cbce4fe04883e72f6f3cac29534ce43ff0b12c7c4e7bc2b5618990e994 not found: ID does not exist"
Sep 08 13:56:31.442812 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.442791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:56:31.446982 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:31.446958    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d9970e-57d5ddd698-nbkd9"]
Sep 08 13:56:33.213520 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:33.213488    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="803c5489-a1df-41e6-ab48-399ccab42ddb" path="/var/lib/kubelet/pods/803c5489-a1df-41e6-ab48-399ccab42ddb/volumes"
Sep 08 13:56:33.378260 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:33.378227    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:56:34.386490 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:34.386461    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:56:37.464822 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.464792    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d1929a2-fcac-4264-9420-24643f9c0628" containerID="d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e" exitCode=0
Sep 08 13:56:37.465217 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.464851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerDied","Data":"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"}
Sep 08 13:56:37.465262 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.465253    2577 scope.go:117] "RemoveContainer" containerID="d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"
Sep 08 13:56:37.466434 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.466401    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:37.466492 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.466479    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:37.466528 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:37.466510    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:56:38.472886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:38.472855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerStarted","Data":"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"}
Sep 08 13:56:38.567960 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:38.567918    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:38.568200 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:38.568180    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 13:56:38.568247 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:38.568226    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 13:56:39.567521 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:39.567481    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 13:56:39.567903 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:39.567541    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 13:56:40.567866 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:40.567828    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 13:56:40.568271 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:40.567886    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 13:56:41.568103 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:41.568067    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 13:56:41.568532 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:41.568124    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 13:56:42.567470 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:42.567438    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 08 13:56:42.567649 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:42.567494    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 08 13:56:43.812246 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:43.812206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:43.812722 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:43.812566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:43.813250 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:43.813236    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 13:56:50.782784 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.782751    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 13:56:50.783195 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783159    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="803c5489-a1df-41e6-ab48-399ccab42ddb" containerName="payload-pre-processing"
Sep 08 13:56:50.783195 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783170    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="803c5489-a1df-41e6-ab48-399ccab42ddb" containerName="payload-pre-processing"
Sep 08 13:56:50.783195 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783190    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783195 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783196    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783204    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783210    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783225    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea4e340-ac55-42bd-9a3e-732b49214a6e" containerName="revoke-keys"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783230    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea4e340-ac55-42bd-9a3e-732b49214a6e" containerName="revoke-keys"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea4e340-ac55-42bd-9a3e-732b49214a6e" containerName="revoke-keys"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783293    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783299    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ac385d3-571f-4d16-86c4-9562c6a73223" containerName="istio-proxy"
Sep 08 13:56:50.783371 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.783304    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="803c5489-a1df-41e6-ab48-399ccab42ddb" containerName="payload-pre-processing"
Sep 08 13:56:50.785503 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.785488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.787804 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.787783    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-cdf4f2\""
Sep 08 13:56:50.787941 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.787815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-cdf4f2-dockercfg-nq5tl\""
Sep 08 13:56:50.796030 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.796008    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 13:56:50.844012 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.843964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9srtl\" (UniqueName: \"kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.844196 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.844086    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.945227 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.945185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9srtl\" (UniqueName: \"kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.945444 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.945264    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.946022 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.945969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:50.952974 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:50.952954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9srtl\" (UniqueName: \"kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl\") pod \"payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:51.097468 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.097374    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:51.374322 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.374294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 13:56:51.376635 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:56:51.376604    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7fe4793_64d5_4f19_a6fb_a04672601583.slice/crio-4a10abfcb6f03bbde30db96b203a6e445b68c48fbbcb355813580694ac4b7046 WatchSource:0}: Error finding container 4a10abfcb6f03bbde30db96b203a6e445b68c48fbbcb355813580694ac4b7046: Status 404 returned error can't find the container with id 4a10abfcb6f03bbde30db96b203a6e445b68c48fbbcb355813580694ac4b7046
Sep 08 13:56:51.573750 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.573712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" event={"ID":"a7fe4793-64d5-4f19-a6fb-a04672601583","Type":"ContainerStarted","Data":"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"}
Sep 08 13:56:51.573930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.573767    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" event={"ID":"a7fe4793-64d5-4f19-a6fb-a04672601583","Type":"ContainerStarted","Data":"4a10abfcb6f03bbde30db96b203a6e445b68c48fbbcb355813580694ac4b7046"}
Sep 08 13:56:51.573930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.573792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:56:51.591800 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:56:51.591744    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" podStartSLOduration=1.591728367 podStartE2EDuration="1.591728367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:51.590635057 +0000 UTC m=+1694.907038630" watchObservedRunningTime="2026-09-08 13:56:51.591728367 +0000 UTC m=+1694.908131920"
Sep 08 13:57:02.581549 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:02.581476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 13:57:19.184122 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:19.184065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 13:57:55.080991 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.080962    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9_cdc3c6c0-6fff-427a-8103-48895e350e37/payload-processing/0.log"
Sep 08 13:57:55.212157 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.212128    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 13:57:55.935877 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.935849    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:57:55.938269 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.938241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:55.940421 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.940397    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-c780a1-openshift-default-dockercfg-q7xgf\""
Sep 08 13:57:55.950739 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:55.950719    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:57:56.092324 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092441    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56xrw\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092602    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.092789 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.092773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193330 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56xrw\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193397    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193462    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.193961 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.194019 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.193995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.194077 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.194020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.194302 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.194278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.195877 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.195852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.196063 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.196047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.202674 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.202650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.202889 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.202867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56xrw\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw\") pod \"e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") " pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.248568 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.248540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:56.525299 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.525273    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:57:56.527020 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:57:56.526976    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbcb77a3_c01e_465c_9c2b_c4c9f212365f.slice/crio-d68775b2f72fc03e22e5c3129d80aef71ec706e8baeaa01e833bdc6b805e092f WatchSource:0}: Error finding container d68775b2f72fc03e22e5c3129d80aef71ec706e8baeaa01e833bdc6b805e092f: Status 404 returned error can't find the container with id d68775b2f72fc03e22e5c3129d80aef71ec706e8baeaa01e833bdc6b805e092f
Sep 08 13:57:56.529456 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.529419    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:57:56.529537 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.529503    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:57:56.529577 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.529545    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:57:56.922531 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.922500    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerStarted","Data":"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"}
Sep 08 13:57:56.922700 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.922547    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerStarted","Data":"d68775b2f72fc03e22e5c3129d80aef71ec706e8baeaa01e833bdc6b805e092f"}
Sep 08 13:57:56.945119 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:56.945074    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podStartSLOduration=1.9450576929999999 podStartE2EDuration="1.945057693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:57:56.94284531 +0000 UTC m=+1760.259248897" watchObservedRunningTime="2026-09-08 13:57:56.945057693 +0000 UTC m=+1760.261461246"
Sep 08 13:57:57.249394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:57.249286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:57.255004 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:57.254977    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:57.927938 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:57.927904    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:57:57.928846 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:57:57.928824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:00.826747 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.826710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:00.829220 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.829204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.831508 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.831490    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-c780a1\""
Sep 08 13:58:00.831615 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.831509    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-c780a1-dockercfg-xfn8x\""
Sep 08 13:58:00.842414 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.842386    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:00.844091 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.844071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnmwd\" (UniqueName: \"kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.844261 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.844238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.944886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.944845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.945084 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.944898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnmwd\" (UniqueName: \"kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.945552 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.945530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:00.953461 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:00.953439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnmwd\" (UniqueName: \"kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:01.140465 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.140438    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:01.417494 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.417466    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:01.418926 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:58:01.418886    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b7e01c5_eed3_4f74_83bc_eedef58d64bc.slice/crio-0c8f091d662a2fba0cd3793d94a12e0651ab5fdec8597a7779d290cc1fba9d4c WatchSource:0}: Error finding container 0c8f091d662a2fba0cd3793d94a12e0651ab5fdec8597a7779d290cc1fba9d4c: Status 404 returned error can't find the container with id 0c8f091d662a2fba0cd3793d94a12e0651ab5fdec8597a7779d290cc1fba9d4c
Sep 08 13:58:01.951886 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.951857    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" event={"ID":"2b7e01c5-eed3-4f74-83bc-eedef58d64bc","Type":"ContainerStarted","Data":"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"}
Sep 08 13:58:01.952270 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.951907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" event={"ID":"2b7e01c5-eed3-4f74-83bc-eedef58d64bc","Type":"ContainerStarted","Data":"0c8f091d662a2fba0cd3793d94a12e0651ab5fdec8597a7779d290cc1fba9d4c"}
Sep 08 13:58:01.952270 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.951962    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:01.970993 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:01.970940    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" podStartSLOduration=1.9709250090000001 podStartE2EDuration="1.970925009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:58:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:01.969558875 +0000 UTC m=+1765.285962448" watchObservedRunningTime="2026-09-08 13:58:01.970925009 +0000 UTC m=+1765.287328562"
Sep 08 13:58:12.959049 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:12.959016    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:13.969563 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:13.969508    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 08 13:58:13.970056 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:13.969573    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 08 13:58:15.038657 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.038560    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerID="b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291" exitCode=0
Sep 08 13:58:15.038657 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.038634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerDied","Data":"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"}
Sep 08 13:58:15.039069 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.039037    2577 scope.go:117] "RemoveContainer" containerID="b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"
Sep 08 13:58:15.040252 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.040224    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:58:15.040331 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.040316    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:58:15.040394 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:15.040368    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 08 13:58:16.046280 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:16.046242    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerStarted","Data":"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"}
Sep 08 13:58:16.249526 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:16.249493    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:16.249779 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:16.249756    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 08 13:58:16.249857 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:16.249839    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 08 13:58:17.249363 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:17.249315    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 08 13:58:17.249746 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:17.249397    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 08 13:58:18.249053 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:18.249016    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 08 13:58:18.249226 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:18.249076    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 08 13:58:19.249310 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:19.249273    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 08 13:58:19.249703 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:19.249365    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 08 13:58:20.249830 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:20.249787    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 08 13:58:20.250223 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:20.249862    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 08 13:58:21.315042 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:21.315013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:21.315590 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:21.315565    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:21.316539 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:21.316520    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:24.610189 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.610150    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"]
Sep 08 13:58:24.614663 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.614638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:24.617159 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.617129    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 13:58:24.623560 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.623538    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"]
Sep 08 13:58:24.681430 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.681395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z\" (UID: \"ca38f876-2c53-432f-8e5e-f9697c7c601e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:24.783023 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.782993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z\" (UID: \"ca38f876-2c53-432f-8e5e-f9697c7c601e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:24.783692 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.783675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z\" (UID: \"ca38f876-2c53-432f-8e5e-f9697c7c601e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:24.927874 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:24.927843    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:25.201731 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:25.201702    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"]
Sep 08 13:58:25.203929 ip-10-0-129-230 kubenswrapper[2577]: W0908 13:58:25.203906    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca38f876_2c53_432f_8e5e_f9697c7c601e.slice/crio-24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b WatchSource:0}: Error finding container 24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b: Status 404 returned error can't find the container with id 24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b
Sep 08 13:58:26.099819 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:26.099782    2577 generic.go:356] "Generic (PLEG): container finished" podID="ca38f876-2c53-432f-8e5e-f9697c7c601e" containerID="a69cdcce02fa8e36bbe979ca489ce923ecc1e91ce0d422a98c0b74c193f68107" exitCode=0
Sep 08 13:58:26.100191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:26.099865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z" event={"ID":"ca38f876-2c53-432f-8e5e-f9697c7c601e","Type":"ContainerDied","Data":"a69cdcce02fa8e36bbe979ca489ce923ecc1e91ce0d422a98c0b74c193f68107"}
Sep 08 13:58:26.100191 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:26.099911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z" event={"ID":"ca38f876-2c53-432f-8e5e-f9697c7c601e","Type":"ContainerStarted","Data":"24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b"}
Sep 08 13:58:27.247096 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:27.247069    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:27.310658 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:27.310627    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca\") pod \"ca38f876-2c53-432f-8e5e-f9697c7c601e\" (UID: \"ca38f876-2c53-432f-8e5e-f9697c7c601e\") "
Sep 08 13:58:27.311025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:27.311002    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ca38f876-2c53-432f-8e5e-f9697c7c601e" (UID: "ca38f876-2c53-432f-8e5e-f9697c7c601e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:27.311083 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:27.311070    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca38f876-2c53-432f-8e5e-f9697c7c601e-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:28.117198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:28.113856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z" event={"ID":"ca38f876-2c53-432f-8e5e-f9697c7c601e","Type":"ContainerDied","Data":"24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b"}
Sep 08 13:58:28.117198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:28.113922    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24ec997a0c45b89eadae9adb914dea08818b445c80492cd635d6e783a339e81b"
Sep 08 13:58:28.117198 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:28.113994    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"
Sep 08 13:58:30.731625 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:30.731589    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:30.732094 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:30.731837    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" podUID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" containerName="payload-pre-processing" containerID="cri-o://502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b" gracePeriod=30
Sep 08 13:58:30.876409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:30.876375    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"]
Sep 08 13:58:30.879229 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:30.879192    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-c780a1-ggt9z"]
Sep 08 13:58:31.009499 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.009474    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:31.130788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.130753    2577 generic.go:356] "Generic (PLEG): container finished" podID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" containerID="502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b" exitCode=0
Sep 08 13:58:31.130788 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.130798    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"
Sep 08 13:58:31.130977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.130844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" event={"ID":"2b7e01c5-eed3-4f74-83bc-eedef58d64bc","Type":"ContainerDied","Data":"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"}
Sep 08 13:58:31.130977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.130892    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k" event={"ID":"2b7e01c5-eed3-4f74-83bc-eedef58d64bc","Type":"ContainerDied","Data":"0c8f091d662a2fba0cd3793d94a12e0651ab5fdec8597a7779d290cc1fba9d4c"}
Sep 08 13:58:31.130977 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.130918    2577 scope.go:117] "RemoveContainer" containerID="502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"
Sep 08 13:58:31.141581 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.141561    2577 scope.go:117] "RemoveContainer" containerID="502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"
Sep 08 13:58:31.141881 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:58:31.141863    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b\": container with ID starting with 502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b not found: ID does not exist" containerID="502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"
Sep 08 13:58:31.141930 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.141892    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b"} err="failed to get container status \"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b\": rpc error: code = NotFound desc = could not find container \"502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b\": container with ID starting with 502c7f9209ecbff212de5186356093df513e674ae47f0aace95fc267bf209a4b not found: ID does not exist"
Sep 08 13:58:31.159254 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.159227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume\") pod \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") "
Sep 08 13:58:31.159409 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.159283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnmwd\" (UniqueName: \"kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd\") pod \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\" (UID: \"2b7e01c5-eed3-4f74-83bc-eedef58d64bc\") "
Sep 08 13:58:31.159653 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.159631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b7e01c5-eed3-4f74-83bc-eedef58d64bc" (UID: "2b7e01c5-eed3-4f74-83bc-eedef58d64bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:31.159732 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.159717    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.161547 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.161526    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd" (OuterVolumeSpecName: "kube-api-access-bnmwd") pod "2b7e01c5-eed3-4f74-83bc-eedef58d64bc" (UID: "2b7e01c5-eed3-4f74-83bc-eedef58d64bc"). InnerVolumeSpecName "kube-api-access-bnmwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:31.213839 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.213809    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca38f876-2c53-432f-8e5e-f9697c7c601e" path="/var/lib/kubelet/pods/ca38f876-2c53-432f-8e5e-f9697c7c601e/volumes"
Sep 08 13:58:31.260641 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.260558    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnmwd\" (UniqueName: \"kubernetes.io/projected/2b7e01c5-eed3-4f74-83bc-eedef58d64bc-kube-api-access-bnmwd\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.446709 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.446675    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:31.450423 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:31.450399    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c780a1-69f788cdb-wdv9k"]
Sep 08 13:58:33.218358 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:33.218315    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" path="/var/lib/kubelet/pods/2b7e01c5-eed3-4f74-83bc-eedef58d64bc/volumes"
Sep 08 13:58:37.284841 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:37.284812    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:58:37.287258 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:37.287235    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 13:58:37.353646 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:37.353618    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:58:37.353905 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:37.353857    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy" containerID="cri-o://b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74" gracePeriod=30
Sep 08 13:58:37.606731 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:37.606653    2577 scope.go:117] "RemoveContainer" containerID="63365c8eeb7dc2774f109eb5f92f074f6c10e63847a002ae76884062b5016d5e"
Sep 08 13:58:42.605286 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.605266    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:42.674092 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674058    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56xrw\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674114    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674226    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674285 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674274    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674324    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674371    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674502 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674457    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket\") pod \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\" (UID: \"dbcb77a3-c01e-465c-9c2b-c4c9f212365f\") "
Sep 08 13:58:42.674762 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674601    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:58:42.674762 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674706    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:42.674888 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674819    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:58:42.674946 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674927    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.675002 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674945    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.675002 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674959    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.675002 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.674994    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:58:42.675323 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.675088    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:58:42.676527 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.676495    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:58:42.676696 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.676675    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 13:58:42.676922 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.676902    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:42.677025 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.676989    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw" (OuterVolumeSpecName: "kube-api-access-56xrw") pod "dbcb77a3-c01e-465c-9c2b-c4c9f212365f" (UID: "dbcb77a3-c01e-465c-9c2b-c4c9f212365f"). InnerVolumeSpecName "kube-api-access-56xrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:42.775855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775821    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.775855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775850    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.775855 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775858    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.776087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775867    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56xrw\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-kube-api-access-56xrw\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.776087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775876    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:42.776087 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:42.775884    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbcb77a3-c01e-465c-9c2b-c4c9f212365f-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:43.196143 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.196115    2577 generic.go:356] "Generic (PLEG): container finished" podID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerID="b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74" exitCode=0
Sep 08 13:58:43.196313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.196178    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"
Sep 08 13:58:43.196313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.196208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerDied","Data":"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"}
Sep 08 13:58:43.196313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.196286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9" event={"ID":"dbcb77a3-c01e-465c-9c2b-c4c9f212365f","Type":"ContainerDied","Data":"d68775b2f72fc03e22e5c3129d80aef71ec706e8baeaa01e833bdc6b805e092f"}
Sep 08 13:58:43.196313 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.196309    2577 scope.go:117] "RemoveContainer" containerID="b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"
Sep 08 13:58:43.207048 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.207025    2577 scope.go:117] "RemoveContainer" containerID="b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"
Sep 08 13:58:43.221838 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.221807    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:58:43.223232 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.223209    2577 scope.go:117] "RemoveContainer" containerID="b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"
Sep 08 13:58:43.223699 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:58:43.223671    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74\": container with ID starting with b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74 not found: ID does not exist" containerID="b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"
Sep 08 13:58:43.223799 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.223711    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74"} err="failed to get container status \"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74\": rpc error: code = NotFound desc = could not find container \"b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74\": container with ID starting with b68a23b8a6008c9b96355e57371420187f781310ab024021acb02f3c4289ca74 not found: ID does not exist"
Sep 08 13:58:43.223799 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.223738    2577 scope.go:117] "RemoveContainer" containerID="b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"
Sep 08 13:58:43.224121 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:58:43.224080    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291\": container with ID starting with b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291 not found: ID does not exist" containerID="b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"
Sep 08 13:58:43.224121 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.224111    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291"} err="failed to get container status \"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291\": rpc error: code = NotFound desc = could not find container \"b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291\": container with ID starting with b75f77eea2174f549964ec2173f55c3cb9d2afcc2bbb9442cc4aa3b1ed841291 not found: ID does not exist"
Sep 08 13:58:43.228593 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:43.228568    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-c780a1-openshift-default-b664b468-6rqr9"]
Sep 08 13:58:45.212671 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:45.212639    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" path="/var/lib/kubelet/pods/dbcb77a3-c01e-465c-9c2b-c4c9f212365f/volumes"
Sep 08 13:58:56.645680 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.645635    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:58:56.646483 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.646448    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" podUID="cdc3c6c0-6fff-427a-8103-48895e350e37" containerName="payload-processing" containerID="cri-o://2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054" gracePeriod=30
Sep 08 13:58:56.656229 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.656201    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:58:56.656948 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.656475    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" podUID="f9643ee2-2bff-463a-8464-607dec62a62b" containerName="payload-pre-processing" containerID="cri-o://1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d" gracePeriod=30
Sep 08 13:58:56.932933 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.932913    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:58:56.935780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:56.935759    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:58:57.013042 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clb2g\" (UniqueName: \"kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g\") pod \"f9643ee2-2bff-463a-8464-607dec62a62b\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") "
Sep 08 13:58:57.013238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013068    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume\") pod \"cdc3c6c0-6fff-427a-8103-48895e350e37\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") "
Sep 08 13:58:57.013238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013150    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume\") pod \"f9643ee2-2bff-463a-8464-607dec62a62b\" (UID: \"f9643ee2-2bff-463a-8464-607dec62a62b\") "
Sep 08 13:58:57.013238 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013214    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52lz8\" (UniqueName: \"kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8\") pod \"cdc3c6c0-6fff-427a-8103-48895e350e37\" (UID: \"cdc3c6c0-6fff-427a-8103-48895e350e37\") "
Sep 08 13:58:57.013592 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013544    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cdc3c6c0-6fff-427a-8103-48895e350e37" (UID: "cdc3c6c0-6fff-427a-8103-48895e350e37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:57.013759 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.013693    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9643ee2-2bff-463a-8464-607dec62a62b" (UID: "f9643ee2-2bff-463a-8464-607dec62a62b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:57.015958 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.015928    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g" (OuterVolumeSpecName: "kube-api-access-clb2g") pod "f9643ee2-2bff-463a-8464-607dec62a62b" (UID: "f9643ee2-2bff-463a-8464-607dec62a62b"). InnerVolumeSpecName "kube-api-access-clb2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:57.016097 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.016074    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8" (OuterVolumeSpecName: "kube-api-access-52lz8") pod "cdc3c6c0-6fff-427a-8103-48895e350e37" (UID: "cdc3c6c0-6fff-427a-8103-48895e350e37"). InnerVolumeSpecName "kube-api-access-52lz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:57.114971 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.114940    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clb2g\" (UniqueName: \"kubernetes.io/projected/f9643ee2-2bff-463a-8464-607dec62a62b-kube-api-access-clb2g\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.114971 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.114965    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cdc3c6c0-6fff-427a-8103-48895e350e37-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.114971 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.114975    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9643ee2-2bff-463a-8464-607dec62a62b-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.114971 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.114985    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52lz8\" (UniqueName: \"kubernetes.io/projected/cdc3c6c0-6fff-427a-8103-48895e350e37-kube-api-access-52lz8\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.270662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.270626    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9643ee2-2bff-463a-8464-607dec62a62b" containerID="1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d" exitCode=0
Sep 08 13:58:57.270662 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.270669    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"
Sep 08 13:58:57.270863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.270717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" event={"ID":"f9643ee2-2bff-463a-8464-607dec62a62b","Type":"ContainerDied","Data":"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"}
Sep 08 13:58:57.270863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.270771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr" event={"ID":"f9643ee2-2bff-463a-8464-607dec62a62b","Type":"ContainerDied","Data":"fff2bfd087dfb346f3d8e4429063342afc6eaebae0369cf5e0836857e42da775"}
Sep 08 13:58:57.270863 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.270799    2577 scope.go:117] "RemoveContainer" containerID="1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"
Sep 08 13:58:57.272795 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.272772    2577 generic.go:356] "Generic (PLEG): container finished" podID="cdc3c6c0-6fff-427a-8103-48895e350e37" containerID="2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054" exitCode=0
Sep 08 13:58:57.273127 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.272973    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"
Sep 08 13:58:57.273226 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.272969    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" event={"ID":"cdc3c6c0-6fff-427a-8103-48895e350e37","Type":"ContainerDied","Data":"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"}
Sep 08 13:58:57.273288 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.273256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9" event={"ID":"cdc3c6c0-6fff-427a-8103-48895e350e37","Type":"ContainerDied","Data":"f82f15a98bd07447f6919c8fb7a5e97a0ecfcb550e1d2f7ef8de995d62582cb7"}
Sep 08 13:58:57.280321 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.280306    2577 scope.go:117] "RemoveContainer" containerID="1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"
Sep 08 13:58:57.280607 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:58:57.280589    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d\": container with ID starting with 1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d not found: ID does not exist" containerID="1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"
Sep 08 13:58:57.280685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.280619    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d"} err="failed to get container status \"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d\": rpc error: code = NotFound desc = could not find container \"1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d\": container with ID starting with 1aee44bef613d53b77b274654bd555099d9e30abd86923078a4632de978bcc9d not found: ID does not exist"
Sep 08 13:58:57.280685 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.280642    2577 scope.go:117] "RemoveContainer" containerID="2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"
Sep 08 13:58:57.288550 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.288535    2577 scope.go:117] "RemoveContainer" containerID="2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"
Sep 08 13:58:57.288729 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.288712    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:58:57.288804 ip-10-0-129-230 kubenswrapper[2577]: E0908 13:58:57.288789    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054\": container with ID starting with 2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054 not found: ID does not exist" containerID="2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"
Sep 08 13:58:57.288852 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.288810    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054"} err="failed to get container status \"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054\": rpc error: code = NotFound desc = could not find container \"2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054\": container with ID starting with 2c01b5406166132f0239543dad94f1bf7ad6411e760063dd777d16a372c59054 not found: ID does not exist"
Sep 08 13:58:57.293034 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.293015    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-2199d0-9db8686f8-5zljr"]
Sep 08 13:58:57.303183 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.303165    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:58:57.307012 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:57.306993    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-2199d0-79df8cd8d7-b9df9"]
Sep 08 13:58:59.212975 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:59.212939    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdc3c6c0-6fff-427a-8103-48895e350e37" path="/var/lib/kubelet/pods/cdc3c6c0-6fff-427a-8103-48895e350e37/volumes"
Sep 08 13:58:59.213403 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:58:59.213298    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9643ee2-2bff-463a-8464-607dec62a62b" path="/var/lib/kubelet/pods/f9643ee2-2bff-463a-8464-607dec62a62b/volumes"
Sep 08 13:59:59.675780 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:59:59.675686    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 13:59:59.676276 ip-10-0-129-230 kubenswrapper[2577]: I0908 13:59:59.675944    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy" containerID="cri-o://21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2" gracePeriod=30
Sep 08 14:00:00.106535 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106453    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"]
Sep 08 14:00:00.106858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106845    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca38f876-2c53-432f-8e5e-f9697c7c601e" containerName="revoke-keys"
Sep 08 14:00:00.106905 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106860    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca38f876-2c53-432f-8e5e-f9697c7c601e" containerName="revoke-keys"
Sep 08 14:00:00.106905 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106872    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdc3c6c0-6fff-427a-8103-48895e350e37" containerName="payload-processing"
Sep 08 14:00:00.106905 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106877    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdc3c6c0-6fff-427a-8103-48895e350e37" containerName="payload-processing"
Sep 08 14:00:00.106905 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106891    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.106905 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106897    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106908    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9643ee2-2bff-463a-8464-607dec62a62b" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106913    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9643ee2-2bff-463a-8464-607dec62a62b" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106925    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106929    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106986    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca38f876-2c53-432f-8e5e-f9697c7c601e" containerName="revoke-keys"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106992    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.106998    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9643ee2-2bff-463a-8464-607dec62a62b" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.107004    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdc3c6c0-6fff-427a-8103-48895e350e37" containerName="payload-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.107011    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7e01c5-eed3-4f74-83bc-eedef58d64bc" containerName="payload-pre-processing"
Sep 08 14:00:00.107082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.107016    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.110103 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.110087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:00.112482 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.112463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 14:00:00.117649 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.117623    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"]
Sep 08 14:00:00.143955 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.143921    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"]
Sep 08 14:00:00.144397 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.144381    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.144498 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.144400    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcb77a3-c01e-465c-9c2b-c4c9f212365f" containerName="istio-proxy"
Sep 08 14:00:00.147563 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.147543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"
Sep 08 14:00:00.148787 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.148750    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"]
Sep 08 14:00:00.151959 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.151942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"
Sep 08 14:00:00.156793 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.156774    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"]
Sep 08 14:00:00.161683 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.161663    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"
Sep 08 14:00:00.166248 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.165979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"]
Sep 08 14:00:00.170200 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.170173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"
Sep 08 14:00:00.219931 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.219694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq\" (UID: \"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:00.321153 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.321118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq\" (UID: \"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:00.321751 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.321731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq\" (UID: \"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:00.422753 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.422716    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:00.448836 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.448804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"]
Sep 08 14:00:00.450011 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:00:00.449977    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce34e5c2_fde2_48f5_82bb_c563a8702538.slice/crio-48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a WatchSource:0}: Error finding container 48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a: Status 404 returned error can't find the container with id 48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a
Sep 08 14:00:00.467633 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.466540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"]
Sep 08 14:00:00.471954 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:00:00.471928    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5062656_5f6c_44b7_8c5f_95955fe04eca.slice/crio-f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21 WatchSource:0}: Error finding container f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21: Status 404 returned error can't find the container with id f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21
Sep 08 14:00:00.604649 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.604614    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9" event={"ID":"a5062656-5f6c-44b7-8c5f-95955fe04eca","Type":"ContainerStarted","Data":"f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21"}
Sep 08 14:00:00.605711 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.605685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn" event={"ID":"ce34e5c2-fde2-48f5-82bb-c563a8702538","Type":"ContainerStarted","Data":"48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a"}
Sep 08 14:00:00.703918 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:00.703892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"]
Sep 08 14:00:00.706988 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:00:00.706965    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e60e2f0_bb03_4add_99f9_034f3d5e0ec0.slice/crio-412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5 WatchSource:0}: Error finding container 412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5: Status 404 returned error can't find the container with id 412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5
Sep 08 14:00:01.613118 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.613077    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" containerID="307cd90915fad1f64677bfc08da3e0c3c99f5de5742487092838b51ef900168e" exitCode=0
Sep 08 14:00:01.613315 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.613158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq" event={"ID":"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0","Type":"ContainerDied","Data":"307cd90915fad1f64677bfc08da3e0c3c99f5de5742487092838b51ef900168e"}
Sep 08 14:00:01.613315 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.613244    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq" event={"ID":"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0","Type":"ContainerStarted","Data":"412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5"}
Sep 08 14:00:01.614746 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.614722    2577 generic.go:356] "Generic (PLEG): container finished" podID="a5062656-5f6c-44b7-8c5f-95955fe04eca" containerID="42902f7dc8eabb5e29775fb77cee7e4491a2e38bc96a801f11b33acee0ca3342" exitCode=0
Sep 08 14:00:01.614855 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.614784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9" event={"ID":"a5062656-5f6c-44b7-8c5f-95955fe04eca","Type":"ContainerDied","Data":"42902f7dc8eabb5e29775fb77cee7e4491a2e38bc96a801f11b33acee0ca3342"}
Sep 08 14:00:01.616296 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.616273    2577 generic.go:356] "Generic (PLEG): container finished" podID="ce34e5c2-fde2-48f5-82bb-c563a8702538" containerID="d8d179f71124e12ed8e18cadc5b3c75ad702e0479e34ad6eeffbc61bed16475d" exitCode=0
Sep 08 14:00:01.616410 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:01.616355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn" event={"ID":"ce34e5c2-fde2-48f5-82bb-c563a8702538","Type":"ContainerDied","Data":"d8d179f71124e12ed8e18cadc5b3c75ad702e0479e34ad6eeffbc61bed16475d"}
Sep 08 14:00:02.778716 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:02.778692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"
Sep 08 14:00:02.824500 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:02.824475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:02.828316 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:02.828289    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"
Sep 08 14:00:02.950694 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:02.950653    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca\") pod \"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0\" (UID: \"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0\") "
Sep 08 14:00:02.951010 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:02.950987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" (UID: "2e60e2f0-bb03-4add-99f9-034f3d5e0ec0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:03.051650 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.051614    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:03.627805 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.627719    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"
Sep 08 14:00:03.627805 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.627726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq" event={"ID":"2e60e2f0-bb03-4add-99f9-034f3d5e0ec0","Type":"ContainerDied","Data":"412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5"}
Sep 08 14:00:03.627805 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.627777    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="412174eb6e710a890dc7872a77b095d3b5d162e4bae00ed6e9ae7c9007f9fac5"
Sep 08 14:00:03.629165 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.629147    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"
Sep 08 14:00:03.629278 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.629147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9" event={"ID":"a5062656-5f6c-44b7-8c5f-95955fe04eca","Type":"ContainerDied","Data":"f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21"}
Sep 08 14:00:03.629278 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.629256    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f039f6b7ca501f0db627915a19b94d28a2a3591e4b190a93302568ad333f0a21"
Sep 08 14:00:03.630745 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.630724    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn"
Sep 08 14:00:03.630880 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.630769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29814600-zmhnn" event={"ID":"ce34e5c2-fde2-48f5-82bb-c563a8702538","Type":"ContainerDied","Data":"48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a"}
Sep 08 14:00:03.630880 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:03.630806    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48c8a24b139d240de1fcaf493a570ce92babd0801fd0339d0458e437a355ab1a"
Sep 08 14:00:04.923702 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.923673    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 14:00:04.970240 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970240 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970209    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970496 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970496 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970263    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970496 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970496 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970325    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970496 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970817 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db4hw\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970900 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data\") pod \"8d1929a2-fcac-4264-9420-24643f9c0628\" (UID: \"8d1929a2-fcac-4264-9420-24643f9c0628\") "
Sep 08 14:00:04.970950 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:04.970950 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970798    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:04.971067 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.970800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:04.971067 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971017    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:04.971067 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:04.971431 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971275    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-credential-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:04.971431 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971294    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-data\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:04.971431 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971305    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-socket\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:04.971431 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971313    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-workload-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:04.971431 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.971329    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d1929a2-fcac-4264-9420-24643f9c0628-istiod-ca-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:04.972841 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.972818    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 14:00:04.973069 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.973045    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:04.973312 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.973288    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw" (OuterVolumeSpecName: "kube-api-access-db4hw") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "kube-api-access-db4hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:04.973312 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:04.973299    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d1929a2-fcac-4264-9420-24643f9c0628" (UID: "8d1929a2-fcac-4264-9420-24643f9c0628"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:00:05.072698 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.072663    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-istio-token\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:05.072698 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.072694    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d1929a2-fcac-4264-9420-24643f9c0628-istio-podinfo\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:05.072698 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.072703    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d1929a2-fcac-4264-9420-24643f9c0628-istio-envoy\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:05.073010 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.072712    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db4hw\" (UniqueName: \"kubernetes.io/projected/8d1929a2-fcac-4264-9420-24643f9c0628-kube-api-access-db4hw\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:05.642149 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.642116    2577 generic.go:356] "Generic (PLEG): container finished" podID="8d1929a2-fcac-4264-9420-24643f9c0628" containerID="21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2" exitCode=0
Sep 08 14:00:05.642328 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.642165    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"
Sep 08 14:00:05.642328 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.642202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerDied","Data":"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"}
Sep 08 14:00:05.642328 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.642259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd" event={"ID":"8d1929a2-fcac-4264-9420-24643f9c0628","Type":"ContainerDied","Data":"34e6dc6c330a7f325a6b9796179da8a4b6ddb79fa5a99ec9618ff0e3d23f41a4"}
Sep 08 14:00:05.642328 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.642285    2577 scope.go:117] "RemoveContainer" containerID="21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"
Sep 08 14:00:05.651666 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.651646    2577 scope.go:117] "RemoveContainer" containerID="d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"
Sep 08 14:00:05.658418 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.658393    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 14:00:05.662382 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.662188    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-2199d0-openshift-default-b8f89ff5b-s2vbd"]
Sep 08 14:00:05.663690 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.663664    2577 scope.go:117] "RemoveContainer" containerID="21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"
Sep 08 14:00:05.664247 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:00:05.664222    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2\": container with ID starting with 21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2 not found: ID does not exist" containerID="21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"
Sep 08 14:00:05.664336 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.664255    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2"} err="failed to get container status \"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2\": rpc error: code = NotFound desc = could not find container \"21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2\": container with ID starting with 21fbf08db123f6a84e5a361b22335b159377662fc0246d17a3e9ae0005393bf2 not found: ID does not exist"
Sep 08 14:00:05.664336 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.664275    2577 scope.go:117] "RemoveContainer" containerID="d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"
Sep 08 14:00:05.664780 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:00:05.664757    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e\": container with ID starting with d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e not found: ID does not exist" containerID="d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"
Sep 08 14:00:05.664858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:05.664787    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e"} err="failed to get container status \"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e\": rpc error: code = NotFound desc = could not find container \"d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e\": container with ID starting with d11173fc3ca2a7b1ee6372b15f76db2fcbb441efc1cd294de9a7d7414358504e not found: ID does not exist"
Sep 08 14:00:07.214829 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:07.214794    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" path="/var/lib/kubelet/pods/8d1929a2-fcac-4264-9420-24643f9c0628/volumes"
Sep 08 14:00:10.646441 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:10.646403    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 14:00:10.646931 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:10.646687    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" podUID="a7fe4793-64d5-4f19-a6fb-a04672601583" containerName="payload-pre-processing" containerID="cri-o://d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db" gracePeriod=30
Sep 08 14:00:10.804525 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:10.804486    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"]
Sep 08 14:00:10.807132 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:10.807101    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-cdf4f2-lrjpq"]
Sep 08 14:00:10.915525 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:10.915498    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 14:00:11.033149 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.033117    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume\") pod \"a7fe4793-64d5-4f19-a6fb-a04672601583\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") "
Sep 08 14:00:11.033392 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.033229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9srtl\" (UniqueName: \"kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl\") pod \"a7fe4793-64d5-4f19-a6fb-a04672601583\" (UID: \"a7fe4793-64d5-4f19-a6fb-a04672601583\") "
Sep 08 14:00:11.033610 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.033583    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7fe4793-64d5-4f19-a6fb-a04672601583" (UID: "a7fe4793-64d5-4f19-a6fb-a04672601583"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:11.035745 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.035683    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl" (OuterVolumeSpecName: "kube-api-access-9srtl") pod "a7fe4793-64d5-4f19-a6fb-a04672601583" (UID: "a7fe4793-64d5-4f19-a6fb-a04672601583"). InnerVolumeSpecName "kube-api-access-9srtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:11.134320 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.134284    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7fe4793-64d5-4f19-a6fb-a04672601583-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:11.134320 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.134315    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9srtl\" (UniqueName: \"kubernetes.io/projected/a7fe4793-64d5-4f19-a6fb-a04672601583-kube-api-access-9srtl\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:11.212910 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.212829    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" path="/var/lib/kubelet/pods/2e60e2f0-bb03-4add-99f9-034f3d5e0ec0/volumes"
Sep 08 14:00:11.679150 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.679121    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7fe4793-64d5-4f19-a6fb-a04672601583" containerID="d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db" exitCode=0
Sep 08 14:00:11.679862 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.679165    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"
Sep 08 14:00:11.679862 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.679199    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" event={"ID":"a7fe4793-64d5-4f19-a6fb-a04672601583","Type":"ContainerDied","Data":"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"}
Sep 08 14:00:11.679862 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.679260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x" event={"ID":"a7fe4793-64d5-4f19-a6fb-a04672601583","Type":"ContainerDied","Data":"4a10abfcb6f03bbde30db96b203a6e445b68c48fbbcb355813580694ac4b7046"}
Sep 08 14:00:11.679862 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.679298    2577 scope.go:117] "RemoveContainer" containerID="d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"
Sep 08 14:00:11.688865 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.688846    2577 scope.go:117] "RemoveContainer" containerID="d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"
Sep 08 14:00:11.689172 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:00:11.689147    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db\": container with ID starting with d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db not found: ID does not exist" containerID="d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"
Sep 08 14:00:11.689219 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.689172    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db"} err="failed to get container status \"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db\": rpc error: code = NotFound desc = could not find container \"d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db\": container with ID starting with d1a4a435a78c5f8142a6a1e7acb5c0e9344568a6c8390b877c7b121d659879db not found: ID does not exist"
Sep 08 14:00:11.693554 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.693529    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 14:00:11.697641 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:11.697616    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-cdf4f2-7764b66bb-nfd6x"]
Sep 08 14:00:13.213667 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:13.213622    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7fe4793-64d5-4f19-a6fb-a04672601583" path="/var/lib/kubelet/pods/a7fe4793-64d5-4f19-a6fb-a04672601583/volumes"
Sep 08 14:00:17.697504 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.697459    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"]
Sep 08 14:00:17.698026 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698011    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5062656-5f6c-44b7-8c5f-95955fe04eca" containerName="cleanup"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698029    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5062656-5f6c-44b7-8c5f-95955fe04eca" containerName="cleanup"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698050    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698068    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce34e5c2-fde2-48f5-82bb-c563a8702538" containerName="cleanup"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698076    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce34e5c2-fde2-48f5-82bb-c563a8702538" containerName="cleanup"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698093    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698101    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698111    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7fe4793-64d5-4f19-a6fb-a04672601583" containerName="payload-pre-processing"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698119    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7fe4793-64d5-4f19-a6fb-a04672601583" containerName="payload-pre-processing"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698130    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" containerName="revoke-keys"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698137    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" containerName="revoke-keys"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698238    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e60e2f0-bb03-4add-99f9-034f3d5e0ec0" containerName="revoke-keys"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698249    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698257    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce34e5c2-fde2-48f5-82bb-c563a8702538" containerName="cleanup"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698267    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5062656-5f6c-44b7-8c5f-95955fe04eca" containerName="cleanup"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698276    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d1929a2-fcac-4264-9420-24643f9c0628" containerName="istio-proxy"
Sep 08 14:00:17.698515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.698288    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7fe4793-64d5-4f19-a6fb-a04672601583" containerName="payload-pre-processing"
Sep 08 14:00:17.703159 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.703137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:17.706027 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.706006    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-44529\""
Sep 08 14:00:17.708004 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.707902    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"]
Sep 08 14:00:17.800182 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.800140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47\" (UID: \"8dccb515-fba2-4533-ab3e-9dfad68bf2ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:17.901562 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.901530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47\" (UID: \"8dccb515-fba2-4533-ab3e-9dfad68bf2ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:17.902220 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:17.902190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47\" (UID: \"8dccb515-fba2-4533-ab3e-9dfad68bf2ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:18.019101 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:18.019016    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:18.291381 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:18.291356    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"]
Sep 08 14:00:18.293322 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:00:18.293290    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dccb515_fba2_4533_ab3e_9dfad68bf2ad.slice/crio-9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06 WatchSource:0}: Error finding container 9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06: Status 404 returned error can't find the container with id 9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06
Sep 08 14:00:18.725404 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:18.725372    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47" event={"ID":"8dccb515-fba2-4533-ab3e-9dfad68bf2ad","Type":"ContainerStarted","Data":"e5b5782ae5a543a8187978962d10527d1514fe13e2aa765b0edc83929a7b36bf"}
Sep 08 14:00:18.725851 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:18.725424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47" event={"ID":"8dccb515-fba2-4533-ab3e-9dfad68bf2ad","Type":"ContainerStarted","Data":"9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06"}
Sep 08 14:00:18.741474 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:18.741419    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47" podStartSLOduration=1.7414027 podStartE2EDuration="1.7414027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:00:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:00:18.73988043 +0000 UTC m=+1902.056283996" watchObservedRunningTime="2026-09-08 14:00:18.7414027 +0000 UTC m=+1902.057806627"
Sep 08 14:00:19.731861 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:19.731830    2577 generic.go:356] "Generic (PLEG): container finished" podID="8dccb515-fba2-4533-ab3e-9dfad68bf2ad" containerID="e5b5782ae5a543a8187978962d10527d1514fe13e2aa765b0edc83929a7b36bf" exitCode=0
Sep 08 14:00:19.732247 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:19.731928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47" event={"ID":"8dccb515-fba2-4533-ab3e-9dfad68bf2ad","Type":"ContainerDied","Data":"e5b5782ae5a543a8187978962d10527d1514fe13e2aa765b0edc83929a7b36bf"}
Sep 08 14:00:20.868396 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:20.868371    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:21.036731 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.036632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca\") pod \"8dccb515-fba2-4533-ab3e-9dfad68bf2ad\" (UID: \"8dccb515-fba2-4533-ab3e-9dfad68bf2ad\") "
Sep 08 14:00:21.037045 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.037008    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8dccb515-fba2-4533-ab3e-9dfad68bf2ad" (UID: "8dccb515-fba2-4533-ab3e-9dfad68bf2ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:21.137654 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.137618    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dccb515-fba2-4533-ab3e-9dfad68bf2ad-maas-api-service-ca\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:21.743855 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.743823    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"
Sep 08 14:00:21.744055 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.743860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47" event={"ID":"8dccb515-fba2-4533-ab3e-9dfad68bf2ad","Type":"ContainerDied","Data":"9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06"}
Sep 08 14:00:21.744055 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:21.743905    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e900e34924e733f0ef005d4f1f2048a4dbea489df09e03d86ac862684af4b06"
Sep 08 14:00:28.601168 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:28.601123    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"]
Sep 08 14:00:28.603157 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:28.603128    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-1f082e-j4g47"]
Sep 08 14:00:29.212945 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:29.212899    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dccb515-fba2-4533-ab3e-9dfad68bf2ad" path="/var/lib/kubelet/pods/8dccb515-fba2-4533-ab3e-9dfad68bf2ad/volumes"
Sep 08 14:00:46.050783 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.050735    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"]
Sep 08 14:00:46.052644 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.052618    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-905fec-29814600-gnhm9"]
Sep 08 14:00:46.071529 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.071503    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 14:00:46.071802 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.071769    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" podUID="955d4192-7edb-4f9d-8842-7c3d09bbef11" containerName="payload-processing" containerID="cri-o://e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a" gracePeriod=30
Sep 08 14:00:46.334301 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.334278    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 14:00:46.481889 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.481850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume\") pod \"955d4192-7edb-4f9d-8842-7c3d09bbef11\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") "
Sep 08 14:00:46.481889 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.481893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hxsj\" (UniqueName: \"kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj\") pod \"955d4192-7edb-4f9d-8842-7c3d09bbef11\" (UID: \"955d4192-7edb-4f9d-8842-7c3d09bbef11\") "
Sep 08 14:00:46.482304 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.482284    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "955d4192-7edb-4f9d-8842-7c3d09bbef11" (UID: "955d4192-7edb-4f9d-8842-7c3d09bbef11"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:46.484156 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.484129    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj" (OuterVolumeSpecName: "kube-api-access-7hxsj") pod "955d4192-7edb-4f9d-8842-7c3d09bbef11" (UID: "955d4192-7edb-4f9d-8842-7c3d09bbef11"). InnerVolumeSpecName "kube-api-access-7hxsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:46.583468 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.583370    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/955d4192-7edb-4f9d-8842-7c3d09bbef11-plugins-config-volume\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.583468 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.583403    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hxsj\" (UniqueName: \"kubernetes.io/projected/955d4192-7edb-4f9d-8842-7c3d09bbef11-kube-api-access-7hxsj\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.891564 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.891523    2577 generic.go:356] "Generic (PLEG): container finished" podID="955d4192-7edb-4f9d-8842-7c3d09bbef11" containerID="e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a" exitCode=0
Sep 08 14:00:46.891765 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.891570    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"
Sep 08 14:00:46.891765 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.891615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" event={"ID":"955d4192-7edb-4f9d-8842-7c3d09bbef11","Type":"ContainerDied","Data":"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"}
Sep 08 14:00:46.891765 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.891676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j" event={"ID":"955d4192-7edb-4f9d-8842-7c3d09bbef11","Type":"ContainerDied","Data":"27637848ee2efc1d0d34a7c17ac779d13d2b681400e5aebbfc6d159a1de9a7af"}
Sep 08 14:00:46.891765 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.891691    2577 scope.go:117] "RemoveContainer" containerID="e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"
Sep 08 14:00:46.901096 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.901079    2577 scope.go:117] "RemoveContainer" containerID="e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"
Sep 08 14:00:46.901404 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:00:46.901382    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a\": container with ID starting with e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a not found: ID does not exist" containerID="e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"
Sep 08 14:00:46.901481 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.901415    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a"} err="failed to get container status \"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a\": rpc error: code = NotFound desc = could not find container \"e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a\": container with ID starting with e912b02fc8b88fbef2c041ec6ebd5357853b89f9fa4fca5fea2712ed3c87da9a not found: ID does not exist"
Sep 08 14:00:46.916223 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.916196    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 14:00:46.920636 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:46.920612    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-905fec-75d7bf86fc-pxk7j"]
Sep 08 14:00:47.214196 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:47.214099    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="955d4192-7edb-4f9d-8842-7c3d09bbef11" path="/var/lib/kubelet/pods/955d4192-7edb-4f9d-8842-7c3d09bbef11/volumes"
Sep 08 14:00:47.214680 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:47.214514    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5062656-5f6c-44b7-8c5f-95955fe04eca" path="/var/lib/kubelet/pods/a5062656-5f6c-44b7-8c5f-95955fe04eca/volumes"
Sep 08 14:00:55.559870 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.559831    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 14:00:55.560295 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.560031    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" podUID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" containerName="manager" containerID="cri-o://3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed" gracePeriod=30
Sep 08 14:00:55.816388 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.816298    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 14:00:55.941293 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.941244    2577 generic.go:356] "Generic (PLEG): container finished" podID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" containerID="3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed" exitCode=0
Sep 08 14:00:55.941511 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.941311    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr"
Sep 08 14:00:55.941511 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.941317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" event={"ID":"d13a3bbe-5932-45ac-9871-9328af4a7e7c","Type":"ContainerDied","Data":"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"}
Sep 08 14:00:55.941511 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.941421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-6wpmr" event={"ID":"d13a3bbe-5932-45ac-9871-9328af4a7e7c","Type":"ContainerDied","Data":"3c502aeefd5e472a673f041f15e6a60b6137d476a9db897b748d612375d49874"}
Sep 08 14:00:55.941511 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.941457    2577 scope.go:117] "RemoveContainer" containerID="3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"
Sep 08 14:00:55.951520 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.951503    2577 scope.go:117] "RemoveContainer" containerID="3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"
Sep 08 14:00:55.951772 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:00:55.951752    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed\": container with ID starting with 3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed not found: ID does not exist" containerID="3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"
Sep 08 14:00:55.951836 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.951785    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed"} err="failed to get container status \"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed\": rpc error: code = NotFound desc = could not find container \"3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed\": container with ID starting with 3f19695375629777938cdf09cec745742c0f601d48847dc126468e35a88555ed not found: ID does not exist"
Sep 08 14:00:55.973132 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.973109    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjbg5\" (UniqueName: \"kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5\") pod \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") "
Sep 08 14:00:55.973245 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.973159    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs\") pod \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") "
Sep 08 14:00:55.973245 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.973191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert\") pod \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\" (UID: \"d13a3bbe-5932-45ac-9871-9328af4a7e7c\") "
Sep 08 14:00:55.975292 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.975268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5" (OuterVolumeSpecName: "kube-api-access-mjbg5") pod "d13a3bbe-5932-45ac-9871-9328af4a7e7c" (UID: "d13a3bbe-5932-45ac-9871-9328af4a7e7c"). InnerVolumeSpecName "kube-api-access-mjbg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:55.975452 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.975433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d13a3bbe-5932-45ac-9871-9328af4a7e7c" (UID: "d13a3bbe-5932-45ac-9871-9328af4a7e7c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:55.975528 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:55.975509    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d13a3bbe-5932-45ac-9871-9328af4a7e7c" (UID: "d13a3bbe-5932-45ac-9871-9328af4a7e7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:56.074121 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:56.074024    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjbg5\" (UniqueName: \"kubernetes.io/projected/d13a3bbe-5932-45ac-9871-9328af4a7e7c-kube-api-access-mjbg5\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:56.074121 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:56.074056    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-metrics-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:56.074121 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:56.074069    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d13a3bbe-5932-45ac-9871-9328af4a7e7c-webhook-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:00:56.262677 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:56.262646    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 14:00:56.268890 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:56.268862    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-6wpmr"]
Sep 08 14:00:57.213331 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:00:57.213288    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" path="/var/lib/kubelet/pods/d13a3bbe-5932-45ac-9871-9328af4a7e7c/volumes"
Sep 08 14:01:57.074312 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074278    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:01:57.074809 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074790    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" containerName="manager"
Sep 08 14:01:57.074858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074812    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" containerName="manager"
Sep 08 14:01:57.074858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074825    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="955d4192-7edb-4f9d-8842-7c3d09bbef11" containerName="payload-processing"
Sep 08 14:01:57.074858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074831    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="955d4192-7edb-4f9d-8842-7c3d09bbef11" containerName="payload-processing"
Sep 08 14:01:57.074858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074845    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dccb515-fba2-4533-ab3e-9dfad68bf2ad" containerName="revoke-keys"
Sep 08 14:01:57.074858 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074851    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dccb515-fba2-4533-ab3e-9dfad68bf2ad" containerName="revoke-keys"
Sep 08 14:01:57.075016 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074908    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dccb515-fba2-4533-ab3e-9dfad68bf2ad" containerName="revoke-keys"
Sep 08 14:01:57.075016 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074919    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d13a3bbe-5932-45ac-9871-9328af4a7e7c" containerName="manager"
Sep 08 14:01:57.075016 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.074925    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="955d4192-7edb-4f9d-8842-7c3d09bbef11" containerName="payload-processing"
Sep 08 14:01:57.078037 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.078019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.080276 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.080252    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 14:01:57.080434 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.080276    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 14:01:57.080589 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.080567    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t9j22\""
Sep 08 14:01:57.080659 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.080607    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 14:01:57.088219 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.088199    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:01:57.157629 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.157591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.157830 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.157653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.157830 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.157759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4cbw\" (UniqueName: \"kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.258590 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.258554    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4cbw\" (UniqueName: \"kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.258772 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.258637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.258772 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.258691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.261781 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.261753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.261922 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.261877    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.267190 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.267161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4cbw\" (UniqueName: \"kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw\") pod \"maas-controller-56cd6bfd4b-2nn5x\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") " pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.389686 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.389643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:01:57.668184 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.668159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:01:57.670181 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:01:57.670154    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae59b53_d909_46c6_bf87_2a2f2b83d5ec.slice/crio-1276ef3f32afd1238f76a5e41777b4d56186f956531d95ef94c3eea42dd0fa7c WatchSource:0}: Error finding container 1276ef3f32afd1238f76a5e41777b4d56186f956531d95ef94c3eea42dd0fa7c: Status 404 returned error can't find the container with id 1276ef3f32afd1238f76a5e41777b4d56186f956531d95ef94c3eea42dd0fa7c
Sep 08 14:01:57.671736 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:57.671712    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:01:58.278837 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:01:58.278804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" event={"ID":"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec","Type":"ContainerStarted","Data":"1276ef3f32afd1238f76a5e41777b4d56186f956531d95ef94c3eea42dd0fa7c"}
Sep 08 14:02:00.290676 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:00.290640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" event={"ID":"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec","Type":"ContainerStarted","Data":"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"}
Sep 08 14:02:00.291080 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:00.290745    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:02:00.308263 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:00.308206    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" podStartSLOduration=1.443747147 podStartE2EDuration="3.308188259s" totalImagesPullingTime="1.864441112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:01:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:02:00.307136145 +0000 UTC m=+2003.623539718" watchObservedRunningTime="2026-09-08 14:02:00.308188259 +0000 UTC m=+2003.624591811"
Sep 08 14:02:11.297735 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:11.297702    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:02:20.687662 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:20.687624    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:02:20.688101 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:20.687835    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" podUID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" containerName="manager" containerID="cri-o://96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7" gracePeriod=30
Sep 08 14:02:20.927848 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:20.927823    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:02:21.097276 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.097183    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4cbw\" (UniqueName: \"kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw\") pod \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") "
Sep 08 14:02:21.097276 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.097244    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs\") pod \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") "
Sep 08 14:02:21.097515 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.097280    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert\") pod \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\" (UID: \"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec\") "
Sep 08 14:02:21.099590 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.099551    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" (UID: "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:02:21.099590 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.099566    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" (UID: "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:02:21.099590 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.099578    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw" (OuterVolumeSpecName: "kube-api-access-c4cbw") pod "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" (UID: "3ae59b53-d909-46c6-bf87-2a2f2b83d5ec"). InnerVolumeSpecName "kube-api-access-c4cbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:02:21.198757 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.198720    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4cbw\" (UniqueName: \"kubernetes.io/projected/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-kube-api-access-c4cbw\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:02:21.198757 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.198751    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-metrics-certs\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:02:21.198757 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.198762    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec-webhook-cert\") on node \"ip-10-0-129-230.ec2.internal\" DevicePath \"\""
Sep 08 14:02:21.399715 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.399675    2577 generic.go:356] "Generic (PLEG): container finished" podID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" containerID="96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7" exitCode=0
Sep 08 14:02:21.399930 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.399727    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x"
Sep 08 14:02:21.399930 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.399720    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" event={"ID":"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec","Type":"ContainerDied","Data":"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"}
Sep 08 14:02:21.399930 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.399797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-2nn5x" event={"ID":"3ae59b53-d909-46c6-bf87-2a2f2b83d5ec","Type":"ContainerDied","Data":"1276ef3f32afd1238f76a5e41777b4d56186f956531d95ef94c3eea42dd0fa7c"}
Sep 08 14:02:21.399930 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.399828    2577 scope.go:117] "RemoveContainer" containerID="96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"
Sep 08 14:02:21.408758 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.408738    2577 scope.go:117] "RemoveContainer" containerID="96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"
Sep 08 14:02:21.409026 ip-10-0-129-230 kubenswrapper[2577]: E0908 14:02:21.409009    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7\": container with ID starting with 96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7 not found: ID does not exist" containerID="96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"
Sep 08 14:02:21.409079 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.409034    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7"} err="failed to get container status \"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7\": rpc error: code = NotFound desc = could not find container \"96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7\": container with ID starting with 96af410eb6658e5cc67fc69fa123e9fe5c5e2db7c8ad254d6546205a9dbfefa7 not found: ID does not exist"
Sep 08 14:02:21.415842 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.415817    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:02:21.422277 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:21.422253    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-2nn5x"]
Sep 08 14:02:22.184323 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.184286    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-fng9p"]
Sep 08 14:02:22.184771 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.184674    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" containerName="manager"
Sep 08 14:02:22.184771 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.184686    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" containerName="manager"
Sep 08 14:02:22.184771 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.184753    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" containerName="manager"
Sep 08 14:02:22.188864 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.188847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.192067 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.192045    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 08 14:02:22.192179 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.192087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 08 14:02:22.192179 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.192088    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-t9j22\""
Sep 08 14:02:22.192279 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.192046    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 08 14:02:22.198608 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.198581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-fng9p"]
Sep 08 14:02:22.206507 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.206481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-metrics-certs\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.206634 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.206523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-webhook-cert\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.206734 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.206715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t695h\" (UniqueName: \"kubernetes.io/projected/4ba4f360-b553-4f27-8e37-6ef26aaef479-kube-api-access-t695h\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.307197 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.307140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-metrics-certs\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.307197 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.307201    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-webhook-cert\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.307456 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.307305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t695h\" (UniqueName: \"kubernetes.io/projected/4ba4f360-b553-4f27-8e37-6ef26aaef479-kube-api-access-t695h\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.309887 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.309862    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-metrics-certs\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.310011 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.309864    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4ba4f360-b553-4f27-8e37-6ef26aaef479-webhook-cert\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.318649 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.318615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t695h\" (UniqueName: \"kubernetes.io/projected/4ba4f360-b553-4f27-8e37-6ef26aaef479-kube-api-access-t695h\") pod \"maas-controller-56cd6bfd4b-fng9p\" (UID: \"4ba4f360-b553-4f27-8e37-6ef26aaef479\") " pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.499743 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.499645    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:22.773960 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:22.773871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56cd6bfd4b-fng9p"]
Sep 08 14:02:22.777077 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:02:22.777050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba4f360_b553_4f27_8e37_6ef26aaef479.slice/crio-94682e2ac979a0269ea3c53a2299b7f5aa778ceb081c73a776cf3250fc7b18b6 WatchSource:0}: Error finding container 94682e2ac979a0269ea3c53a2299b7f5aa778ceb081c73a776cf3250fc7b18b6: Status 404 returned error can't find the container with id 94682e2ac979a0269ea3c53a2299b7f5aa778ceb081c73a776cf3250fc7b18b6
Sep 08 14:02:23.213623 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:23.213595    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ae59b53-d909-46c6-bf87-2a2f2b83d5ec" path="/var/lib/kubelet/pods/3ae59b53-d909-46c6-bf87-2a2f2b83d5ec/volumes"
Sep 08 14:02:23.412037 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:23.412006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p" event={"ID":"4ba4f360-b553-4f27-8e37-6ef26aaef479","Type":"ContainerStarted","Data":"b8871ce89981ebab90ac580837cf0065ada145b0c5a26cfb09f4379f1f5bcb01"}
Sep 08 14:02:23.412184 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:23.412054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p" event={"ID":"4ba4f360-b553-4f27-8e37-6ef26aaef479","Type":"ContainerStarted","Data":"94682e2ac979a0269ea3c53a2299b7f5aa778ceb081c73a776cf3250fc7b18b6"}
Sep 08 14:02:23.412184 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:23.412085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:23.432080 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:23.431965    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p" podStartSLOduration=1.034394956 podStartE2EDuration="1.431944652s" totalImagesPullingTime="397.549696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:02:23.430249178 +0000 UTC m=+2026.746652748" watchObservedRunningTime="2026-09-08 14:02:23.431944652 +0000 UTC m=+2026.748348204"
Sep 08 14:02:34.420325 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:34.420285    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56cd6bfd4b-fng9p"
Sep 08 14:02:37.685600 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:37.685573    2577 scope.go:117] "RemoveContainer" containerID="506f4cbf0364191c5a3e9301db03168eb668d8bcc94144176c65b7e62e72882a"
Sep 08 14:02:37.696576 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:37.696553    2577 scope.go:117] "RemoveContainer" containerID="fec05674ba9eac30da3c7aff369703eb2e3fee766ababd1dba52f9d7c6dc3298"
Sep 08 14:02:37.704800 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:02:37.704778    2577 scope.go:117] "RemoveContainer" containerID="5d784bcb82bc1762ef951a02d47f3e64449c1386b9aaeb5883ab9a50a88aeb18"
Sep 08 14:03:37.317782 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:03:37.317748    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 14:03:37.321895 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:03:37.321874    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 14:04:37.742630 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:04:37.742597    2577 scope.go:117] "RemoveContainer" containerID="a69cdcce02fa8e36bbe979ca489ce923ecc1e91ce0d422a98c0b74c193f68107"
Sep 08 14:06:37.769356 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:06:37.769303    2577 scope.go:117] "RemoveContainer" containerID="e5b5782ae5a543a8187978962d10527d1514fe13e2aa765b0edc83929a7b36bf"
Sep 08 14:06:37.778128 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:06:37.778105    2577 scope.go:117] "RemoveContainer" containerID="42902f7dc8eabb5e29775fb77cee7e4491a2e38bc96a801f11b33acee0ca3342"
Sep 08 14:06:37.786322 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:06:37.786297    2577 scope.go:117] "RemoveContainer" containerID="307cd90915fad1f64677bfc08da3e0c3c99f5de5742487092838b51ef900168e"
Sep 08 14:07:03.521739 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:03.521705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56cd6bfd4b-fng9p_4ba4f360-b553-4f27-8e37-6ef26aaef479/manager/0.log"
Sep 08 14:07:03.896187 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:03.896161    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-hl2bt_de1c6466-e64c-448c-86c0-07b45e9e8beb/copy-manifests/0.log"
Sep 08 14:07:03.902984 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:03.902963    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-hl2bt_de1c6466-e64c-448c-86c0-07b45e9e8beb/manager/0.log"
Sep 08 14:07:04.777003 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:04.776966    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814585-xm6tg_50f0c29d-817e-4e10-9f78-5b6cf6d59fcf/cleanup/0.log"
Sep 08 14:07:04.901635 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:04.901602    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29814600-zmhnn_ce34e5c2-fde2-48f5-82bb-c563a8702538/cleanup/0.log"
Sep 08 14:07:05.020283 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:05.020255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-jjsgk_5b0740d8-33c7-4d83-bb6c-fb21a656448a/postgres/0.log"
Sep 08 14:07:07.050194 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.050164    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl_879b6174-5433-4b2b-bc79-fb4ae24e6d36/istio-proxy/0.log"
Sep 08 14:07:07.173126 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.173094    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9cxwg_19097394-d232-4cb8-a149-500bff74505b/discovery/0.log"
Sep 08 14:07:07.570962 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.570933    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/1.log"
Sep 08 14:07:07.696020 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.695991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cx6dt_6314af19-a498-4ab0-8110-b5156bc17d3f/payload-pre-processing/0.log"
Sep 08 14:07:07.839375 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.839263    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 14:07:07.994368 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:07.994323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f56957bd6-hjkvs_3b045e1d-7b5b-4cf8-b851-56b866652bef/router/0.log"
Sep 08 14:07:08.617319 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:08.617290    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-zs5s8_d6f0c547-a747-4a48-b3cc-19545f9f4786/main/0.log"
Sep 08 14:07:08.736274 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:08.736243    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-78r5h_d001b502-050b-4a2b-b44a-2a0b4fb6cc36/main/0.log"
Sep 08 14:07:08.975217 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:08.975188    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-fvxfg_51fed2b7-b88f-4ac4-aa49-4c55f3e1edb4/main/0.log"
Sep 08 14:07:17.637022 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:17.636984    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pv4qm_1933aed9-a5f1-4d70-957b-80c97eddfe96/global-pull-secret-syncer/0.log"
Sep 08 14:07:17.788215 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:17.788187    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xr4gh_25a6cdf5-7ff9-45f2-9c47-311be2aeeec9/konnectivity-agent/0.log"
Sep 08 14:07:17.810978 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:17.810950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-230.ec2.internal_099d282a8bec5dd8dd074b5f725a2d0c/haproxy/0.log"
Sep 08 14:07:24.442279 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.442247    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-j69xv_f24ddbfa-f5f9-48f2-8664-a591f3f2fd18/cluster-monitoring-operator/0.log"
Sep 08 14:07:24.555639 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.555610    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-684b46cf8f-ps7wt_d50f17c3-94d1-492d-af02-24441997414e/metrics-server/0.log"
Sep 08 14:07:24.714971 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.714942    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prqgm_79d15551-ba67-4b0f-9fb6-c8148e502eca/node-exporter/0.log"
Sep 08 14:07:24.742538 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.742518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prqgm_79d15551-ba67-4b0f-9fb6-c8148e502eca/kube-rbac-proxy/0.log"
Sep 08 14:07:24.767390 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.767364    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-prqgm_79d15551-ba67-4b0f-9fb6-c8148e502eca/init-textfile/0.log"
Sep 08 14:07:24.942182 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.942155    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tgvls_1a0099cf-450c-4191-8872-58abc5a6e261/kube-rbac-proxy-main/0.log"
Sep 08 14:07:24.971578 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:24.971490    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tgvls_1a0099cf-450c-4191-8872-58abc5a6e261/kube-rbac-proxy-self/0.log"
Sep 08 14:07:25.005618 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.005594    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-tgvls_1a0099cf-450c-4191-8872-58abc5a6e261/openshift-state-metrics/0.log"
Sep 08 14:07:25.057776 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.057750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/prometheus/0.log"
Sep 08 14:07:25.087449 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.087423    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/config-reloader/0.log"
Sep 08 14:07:25.111549 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.111527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/thanos-sidecar/0.log"
Sep 08 14:07:25.137646 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.137610    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/kube-rbac-proxy-web/0.log"
Sep 08 14:07:25.159476 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.159449    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/kube-rbac-proxy/0.log"
Sep 08 14:07:25.181767 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.181746    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/kube-rbac-proxy-thanos/0.log"
Sep 08 14:07:25.208063 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.208038    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af3629df-4ff7-49ac-9a10-6c8775c3fd5f/init-config-reloader/0.log"
Sep 08 14:07:25.237976 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.237897    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gcnxd_9575bc9c-bbc7-4a5f-a406-3275555fd8ca/prometheus-operator/0.log"
Sep 08 14:07:25.258948 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.258923    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-gcnxd_9575bc9c-bbc7-4a5f-a406-3275555fd8ca/kube-rbac-proxy/0.log"
Sep 08 14:07:25.285729 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:25.285705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-5c67d_9ca704a8-1f03-4ed1-aae6-0f6eb9b9420d/prometheus-operator-admission-webhook/0.log"
Sep 08 14:07:27.660023 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:27.659997    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/2.log"
Sep 08 14:07:27.664602 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:27.664582    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-q2shb_0eb9e039-751e-495a-afad-fd5f8ef829c7/console-operator/3.log"
Sep 08 14:07:28.880092 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:28.880015    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-5894r_a571c7c2-e643-490f-9015-04851cb33359/volume-data-source-validator/0.log"
Sep 08 14:07:29.183018 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.182981    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"]
Sep 08 14:07:29.186664 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.186648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.188965 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.188936    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52jtx\"/\"kube-root-ca.crt\""
Sep 08 14:07:29.188965 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.188940    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52jtx\"/\"openshift-service-ca.crt\""
Sep 08 14:07:29.189538 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.189517    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-52jtx\"/\"default-dockercfg-dxk4h\""
Sep 08 14:07:29.195011 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.194990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"]
Sep 08 14:07:29.258532 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.258495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-sys\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.258705 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.258613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvcb9\" (UniqueName: \"kubernetes.io/projected/e2a762f3-73fc-4bc2-af44-68df15b48ba8-kube-api-access-gvcb9\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.258705 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.258690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-lib-modules\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.258809 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.258732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-proc\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.258926 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.258822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-podres\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.359797 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-lib-modules\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.359797 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-proc\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-podres\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-sys\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvcb9\" (UniqueName: \"kubernetes.io/projected/e2a762f3-73fc-4bc2-af44-68df15b48ba8-kube-api-access-gvcb9\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-proc\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-lib-modules\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.359997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-sys\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.360074 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.360031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a762f3-73fc-4bc2-af44-68df15b48ba8-podres\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.368516 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.368483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvcb9\" (UniqueName: \"kubernetes.io/projected/e2a762f3-73fc-4bc2-af44-68df15b48ba8-kube-api-access-gvcb9\") pod \"perf-node-gather-daemonset-qxrgp\" (UID: \"e2a762f3-73fc-4bc2-af44-68df15b48ba8\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.498112 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.498019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:29.770116 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.770092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"]
Sep 08 14:07:29.772471 ip-10-0-129-230 kubenswrapper[2577]: W0908 14:07:29.772444    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode2a762f3_73fc_4bc2_af44_68df15b48ba8.slice/crio-a487c517d12e9df627a42524363d728aa7e7858eaa9dcbc01a433036b8af91ca WatchSource:0}: Error finding container a487c517d12e9df627a42524363d728aa7e7858eaa9dcbc01a433036b8af91ca: Status 404 returned error can't find the container with id a487c517d12e9df627a42524363d728aa7e7858eaa9dcbc01a433036b8af91ca
Sep 08 14:07:29.986698 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.986661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp" event={"ID":"e2a762f3-73fc-4bc2-af44-68df15b48ba8","Type":"ContainerStarted","Data":"184a54c94ca8289b7e395beb1c7df7f0f302efa49c60385ab6463912c59369f5"}
Sep 08 14:07:29.987095 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.986711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp" event={"ID":"e2a762f3-73fc-4bc2-af44-68df15b48ba8","Type":"ContainerStarted","Data":"a487c517d12e9df627a42524363d728aa7e7858eaa9dcbc01a433036b8af91ca"}
Sep 08 14:07:29.987095 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:29.986797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:30.004051 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:30.004002    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp" podStartSLOduration=1.003985362 podStartE2EDuration="1.003985362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:07:30.001946367 +0000 UTC m=+2333.318349937" watchObservedRunningTime="2026-09-08 14:07:30.003985362 +0000 UTC m=+2333.320388976"
Sep 08 14:07:30.189494 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:30.189460    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn66g_98b7648c-e184-4582-a115-a72957ac4884/dns/0.log"
Sep 08 14:07:30.209687 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:30.209644    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nn66g_98b7648c-e184-4582-a115-a72957ac4884/kube-rbac-proxy/0.log"
Sep 08 14:07:30.340509 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:30.340475    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z4kt5_1f28bf03-0013-4c8b-a313-6aef862af55f/dns-node-resolver/0.log"
Sep 08 14:07:30.927752 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:30.927722    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5dpqb_bb16b466-0c5c-4c93-a0ba-d83777864ac7/node-ca/0.log"
Sep 08 14:07:31.923615 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:31.923588    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl_879b6174-5433-4b2b-bc79-fb4ae24e6d36/istio-proxy/0.log"
Sep 08 14:07:32.209503 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.209426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9cxwg_19097394-d232-4cb8-a149-500bff74505b/discovery/0.log"
Sep 08 14:07:32.289072 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.289047    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/0.log"
Sep 08 14:07:32.297825 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.297804    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/1.log"
Sep 08 14:07:32.331832 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.331800    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cx6dt_6314af19-a498-4ab0-8110-b5156bc17d3f/payload-pre-processing/0.log"
Sep 08 14:07:32.367035 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.367007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 14:07:32.390285 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:32.390258    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f56957bd6-hjkvs_3b045e1d-7b5b-4cf8-b851-56b866652bef/router/0.log"
Sep 08 14:07:33.325647 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:33.325612    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n5nwm_07486108-6f0e-4822-a7bd-aedfc71b4112/serve-healthcheck-canary/0.log"
Sep 08 14:07:33.825355 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:33.825326    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sfvwn_15c878ca-0372-4bc5-b902-625461929dbf/kube-rbac-proxy/0.log"
Sep 08 14:07:33.850879 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:33.850854    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sfvwn_15c878ca-0372-4bc5-b902-625461929dbf/exporter/0.log"
Sep 08 14:07:33.873561 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:33.873533    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sfvwn_15c878ca-0372-4bc5-b902-625461929dbf/extractor/0.log"
Sep 08 14:07:36.003868 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:36.003837    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-qxrgp"
Sep 08 14:07:36.484612 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:36.484584    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56cd6bfd4b-fng9p_4ba4f360-b553-4f27-8e37-6ef26aaef479/manager/0.log"
Sep 08 14:07:36.614008 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:36.613982    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-hl2bt_de1c6466-e64c-448c-86c0-07b45e9e8beb/manager/0.log"
Sep 08 14:07:36.641729 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:36.641704    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-hl2bt_de1c6466-e64c-448c-86c0-07b45e9e8beb/copy-manifests/0.log"
Sep 08 14:07:38.053630 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:38.053607    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8d99975cf-kz9cb_9268d61d-0852-4e4e-bc67-6a6126763720/manager/0.log"
Sep 08 14:07:38.878082 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:38.878043    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl_879b6174-5433-4b2b-bc79-fb4ae24e6d36/istio-proxy/0.log"
Sep 08 14:07:39.170424 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.170390    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9cxwg_19097394-d232-4cb8-a149-500bff74505b/discovery/0.log"
Sep 08 14:07:39.227901 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.227870    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/0.log"
Sep 08 14:07:39.237447 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.237420    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/1.log"
Sep 08 14:07:39.263333 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.263300    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cx6dt_6314af19-a498-4ab0-8110-b5156bc17d3f/payload-pre-processing/0.log"
Sep 08 14:07:39.289080 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.289052    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 14:07:39.304973 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:39.304947    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f56957bd6-hjkvs_3b045e1d-7b5b-4cf8-b851-56b866652bef/router/0.log"
Sep 08 14:07:40.851036 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:40.851007    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-dbvbl_879b6174-5433-4b2b-bc79-fb4ae24e6d36/istio-proxy/0.log"
Sep 08 14:07:41.133511 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.133426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9cxwg_19097394-d232-4cb8-a149-500bff74505b/discovery/0.log"
Sep 08 14:07:41.193555 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.193527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/0.log"
Sep 08 14:07:41.202457 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.202420    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-vtl2w_0460699c-2133-46e6-b289-148b07e9ce69/istio-proxy/1.log"
Sep 08 14:07:41.231883 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.231854    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-cx6dt_6314af19-a498-4ab0-8110-b5156bc17d3f/payload-pre-processing/0.log"
Sep 08 14:07:41.256853 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.256820    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-tc5vb_8f7e94ea-a9f6-4811-b42f-98fd19a7bcd8/payload-processing/0.log"
Sep 08 14:07:41.275244 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:41.275226    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f56957bd6-hjkvs_3b045e1d-7b5b-4cf8-b851-56b866652bef/router/0.log"
Sep 08 14:07:43.692658 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:43.692623    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-gmgf5_e1d592c7-9383-4352-b307-fdd687269c5c/migrator/0.log"
Sep 08 14:07:43.714589 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:43.714551    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-gmgf5_e1d592c7-9383-4352-b307-fdd687269c5c/graceful-termination/0.log"
Sep 08 14:07:44.090016 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:44.089896    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pmllc_61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8/kube-storage-version-migrator-operator/1.log"
Sep 08 14:07:44.090869 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:44.090849    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pmllc_61a5ebb7-c1d0-4ad5-b6f4-0980076eeec8/kube-storage-version-migrator-operator/0.log"
Sep 08 14:07:45.571333 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.571240    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/kube-multus-additional-cni-plugins/0.log"
Sep 08 14:07:45.593773 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.593727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/egress-router-binary-copy/0.log"
Sep 08 14:07:45.615246 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.615203    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/cni-plugins/0.log"
Sep 08 14:07:45.638776 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.638746    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/bond-cni-plugin/0.log"
Sep 08 14:07:45.663637 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.663617    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/routeoverride-cni/0.log"
Sep 08 14:07:45.684595 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.684571    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/whereabouts-cni-bincopy/0.log"
Sep 08 14:07:45.706896 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.706867    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtj7f_6da93b03-2b9d-4eab-a92f-d5ce3f5a8219/whereabouts-cni/0.log"
Sep 08 14:07:45.747890 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.747861    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7wk8_eea60f6f-0516-43d2-b6c3-6f7c24fbc098/kube-multus/0.log"
Sep 08 14:07:45.812673 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.812644    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jvplh_882c9adf-dd13-46b6-9471-dbafc81deb3c/network-metrics-daemon/0.log"
Sep 08 14:07:45.833298 ip-10-0-129-230 kubenswrapper[2577]: I0908 14:07:45.833231    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jvplh_882c9adf-dd13-46b6-9471-dbafc81deb3c/kube-rbac-proxy/0.log"